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THE FIRST systematic attempt to apply research and development 
strategies to education occurred with the establishment of four research and 
development (В & D) centers in 1964 as part of the Cooperative Research Pro- 
gram of the United States Office of Education. The R & D Centers program attempted 
to combine the best features of the separate programs of the U.S.O.E.: basic 
and applied research, development, curriculum improvement, and demonstration. 
Thus, an R&D Center is organized to conduct basic research related to a sig- 
nificant problem area; to develop substantive and procedural products and con- 
duct related research; to demonstrate the results of the research and develop- 
ment activities in a few school settings; and to take initiative for the eventual 
widespread dissemination of the research results and tested products. An R& D 
Center, located on a university campus, draws its professorial staff from the be- 
havioral sciences, subject disciplines, communications, educational methodology, 
and other fields. It is a community of scholars, capable of dealing with a prob- 
lem area effectively. In an R &D Center, then, human and financial resources 


Our major universities, including the nine where R& D Centers are 
located, have a large pool of able researchers and many other resources, How- 
ever, the development and application of research and development strategies to 
education is a pioneer venture. Developing imaginative strategies for extending 
knowledge about cognitive learning and improving related educational practices 
is a continuing concern of the Wisconsin Research and Development Center for 
Cognitive Learning. 

In this Center, basic research is done to extend knowledge about 
the processes of cognitive learning, particularly those associated with concept 
learning and problem solving, and about the conditions associated with efficiency 
of learning in the cognitive domain. Some of this knowledge goes immediately 
into the development of instructional systems. A second objective of the Wiscon- 
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sin R&D Center is the improvement of educational practices in the cognitive 
domain. The development and refinement of an instructional system т 
a subject field, such as mathematics, requires appropriate strategies for dea E 
with the various components independently and also for interrelating them “ 
that desired behavioral objectives are achieved. Three important components T 
the system are subject matter content and sequence; instructional materials an 
media; and instructional methods, including teacher behaviors and learner activi- 
ties. A colloquium series was organized to bring information to the Center staff 
regarding these research and development strategies. 

One part of this colloquium series was concerned with research on 
learning that has fairly direct implications for improving education. (A pins 
volume, Analyses of Concept Learning, Academic Press, 1966, on the other han 
outlined more basic research.) The main portion of the colloquium series was 
addressed to development and related research dealing with subject matter con- 
tent and sequence, instructional media, and instructional methods. This volume 
is organized in the same pattern. | 

A As might be expected, the research methods reported by the partici- 
| pants are varied. In Part 1, Endel Tulving of the University of Toronto shows that 

! substantive results concerning information storage and retrieval, derived from the 
I typical controlled laboratory experiment, require us to reconsider our notions about 
why persons do not effectively reproduce information they once learned. Kenneth 
Lovell, University of Leeds, shows that the charting of thought processes develop- 
mentally is moving ahead rapidly by means of Piaget’s methods and refined vari- 
ants thereof. Martin Covington, University of California, describes a straightfor- 


ward attempt to develop materials designed to facilitate productive thinking i? 
school children and to utilize the material in extending knowledge about chil- 
Ше: productive thinking skills. Here there is a continuous interweaving of theo! 
izing, developing material based on theory, and research activities concerning 
the abilities nurtured by the instructional program. Richard DeCharms and Vit 
gug Carpenter of Washington University and Lindenwood College proceed in a 
similiar manner as Covington; however, their main concern is with motivation? 
Pise and the measurement of strength of motivation in culturally disadvantage 
children. In these chapters substantive research results and theory properly receive 
most attention; however, sufficient attention is given to the procedures to та E 


clear that extending knowledge about processes and conditions of learning ca" 

be accomplished by a variety of strategies. h 

pracedore: In Patt ll, somewhat similar curriculum development and paa 

een, ў outlined by the first three authors. Е. G. Begle, Чаш HM a 

erie ез evelopment and related research in mathematics, including his e m 

AA @ WI The widely known School Mathematics Study Group. Arthur Шуегто' 
AS Commissi 


2 x E à ^ cU 
9n on Science Education, briefly describes the large science 
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riculum projects, kindergarten to Grade 12, but gives major attention to the 
elementary program: Science—A Process Approach. Robert C. Pooley, The Uni- 
versity of Wisconsin, also synthesizes many research and development activities 
related to the teaching of literature, composition, and language but gives par- 
ticular attention to the Wisconsin English-Language Arts Project. Common to the 
curriculum development strategies in these subjects is an increasing emphasis on 
the structure of the discipline and a decreasing emphasis on other criteria such 
as social utility, difficulty, children's interests, and possible transfer to other sub- 
ject fields. In the last two chapters of Part Il, Wayne Otto, The University of Wis- 
consin, describes a rigorous and productive research effort in reading, and Myron 
Rosskopf, Teachers College, Columbia University, relates curriculum development 
in mathematics to recent research in concept learning. 

In Part 111, Wilbur Schramm, Stanford University, reports information 
on the use of instructional television. Three case studies are presented in suffi- 
cient detail to renew confidence in the value of careful observation in the natural 
environment by an interested, objective scholar. Lawrence Stolurow, Harvard Uni- 
versity, describes the computer-based instructional system, SOCRATES, that he 
developed while at the University of Illinois. Problems associated with develop- 
ment of the hardware and software are noted briefly or may be inferred. The use 
of the computer in research to improve instruction and to validate behavior theory 
is noteworthy. Arthur Lumsdaine, University of Washington, deals with research 
and development in the entire field of instructional media. 

In Part IV, N. L. Gage, Stanford University, outlines a recent con- 
cept, micro-teaching, and related research strategies. The concept promises to 
generate much research and development on instructional methods. In this approach, 
the teacher and student rather than the content, hardware, or program, are 
central in the educative Process. Ned Flanders, University of Michigan, describes 
the new directions of his long-standing research on teacher-student interaction anal- 
ysis. In the current model, the number of categories of interactions is reduced, By 
employing a computer for rapid feedback, the current model holds much promise 
of providing immediate (daily) correction of teacher—student interactions and also 
for supplying the extensive research data required in validating and extending 
the theoretical basis of the system of interaction analysis. 

Part V of this volume contains papers which did not arise from the 
colloquium series. They are included because they contribute to the survey of 
research and development activities in education, particularly very recent trends. 
louis Bright and Hendrick Gideonse of the U.S.O.E. outline the broad sweep of 
research and development strategies and activities, resulting from the Passage 
of the Elementary and Secondary Education Act of 1965. Included here are the 
R& D centers, regional educational laboratories, Title III centers, and other R & р 
'Nstrumentalities funded by other governmental agencies. The comprehensive edy- 
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cational research and development program of the U.S.O.E. merits careful atten- 
tion by anyone interested in the improvement of education. A more complete 
description of one federally-supported R & D instrumentality, the Wisconsin Re- 
search and Development Center for Cognitive Learning, follows the chapter by 
Bright and Gideonse. Here we review the main components of a Center and their 
interrelations. Particular attention is given to the problem area of the Center; 
its programs, strategies, and outcomes; staffing pattern; administrative organi- 
zation; locus of operation; and source and stability of funding. 

In a new field, models are required as guides to operations. Many 
models have been outlined to indicate how systematic improvement in education 
might be managed through sequential research, development, and dissemination. 
None of the models thus far appeared particularly appropriate to the current scene. 
Hendrick Gideonse, active privately and not as a U.S.O.E. official, has been 
attempting for some time to formulate a comprehensive model that would simul- 
taneously meet the test of reality and also provide a more adequate conceptuali- 
zation of Research and Development activities designed to improve education. 
Fortunately his conceptualization was completed at the time the volume was in 
final editing. The Gideonse model is especially relevant to those who have been 
concerned with relating research, development, and educational improvement. 

The colloquium series was planned and executed by the editors of this 
volume; Herbert J. Klausmeier assumed primary responsibility for organizing the 
colloquium series around topical areas of relevance to the R & D Center. He identi- 
fled most of the participants and the broad organization framework for each par- 
ticipant's contribution. George T. O'Hearn participated in the planning and organi- 
zation of the colloquium series, worked out the presentation schedule and оће! 
details with each participant, and handled editorial queries from the publishe™ 
He also edited the papers and wrote the introductions to each part. Barbare? 
Kennedy, Project Assistant, read the papers, checked the references, and pul 
the references in APA style. The Wisconsin Research and Development Center ig 
Cognitive Learning, the education research and development community, 9" 


the teaching profession generally is indebted to the internationally recogniZ* 
scholars who made this volume possible. 


Herbert J. Klausmeier, Professor of 
Educational Psychology; Director, 
Wisconsin Research and Developme" 
Center for Cognitive Learning 
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iil! Conditions and Processes of Learning 


UNDERSTANDING learning phenomena and conditions associated 
with efficiency of learning is a foremost concern of psychologists throughout West- 
ern civilization. Increasingly sophisticated experiments are being executed in labo- 
ratory and non-laboratory settings. Representative of work in this vast domain, 
with implications for education, are the following: 

a) controlled observations to infer thought processes. 
b) controlled experimentation in the laboratory to extend our knowledge of 
human memory. 


c) development of material and related research to nurture the creative abili- 
ties of school children. 

d) development of dependent measures to extend a motivational theory to 
school settings. 

Permanence of learning necessarily requires the storage and recall 
of information. In Chapter 1, Endel Tulving discusses various verbal learning and 
memory experiments conducted in the laboratory at the University of Toronto, 
and suggests that "the success of retention and recall is to a large extent deter- 
mined by what the subject does with the material he is to memorize, by the meth- 
ods he uses to organize the material, and by the cues that he uses to effect the 
recall of the material." The data reported by Tulving support his contention that 
the ability to recall is determined more by the limitation of the retrieval mechanism 
than by the limitation of the storage mechanism. He concludes: 

. that understanding of many phenomena of verbal learning and memory would 
be enhanced if we adopted a basic orienting attitude. That a person's mnemonic perform. 
ance depends to a very large extent on the method that he wittingly or unwittingly 
Uses fo organize the material. The important function of such organization is to facili- 
tate retrieval of information from the memory storage. 

The inferring of thought processes has always been difficult but is 
essential to explaining changes that occur ontogenetically. In "Developmental Proc- 
esses in Thought," Kenneth Lovell, University of Leeds, reports on the stages of 


Min ЕЕ 


human intellectual growth hypothesized by Jean Piaget and his followers. Lovell 
suggests that the schemas interiorized by the child undergo constant secre 
tion and are the basis and limitation of all subsequent learning. ‘The implication K 
Piaget's findings is that we need to begin to stimulate the intellectual growth ie 
the child in the early weeks of life." Research reported by Lovell confirms t 
general classification of the developmental stages of thought hypothesized а. 
Piaget, and also indicates the importance of previous learning and the cultura 
milieu on the child. 

Martin Covington in his paper ‘Promoting Creative Thinking in the 
' states, "In order to take full advantage of the unrestrictive atmos” 
phere the child must first come to understand what constitutes creative ideas i" 


the given situation, and how he can achieve such ideas for himself . . . In огде" 
to think creatively the child must have at his disposal a repertoire of creative think 
ing skills. To name a few: the ability to recognize gaps in existing information; 4 
facility for formulating relevant questions, and a facility to the demands of the 
task so one can adequately judge the suitability of the proposed ideas." 

The General Problem Solving Program described by Covington repre” 
sents an attempt to initiate creative thought in students within a school setting 
and within a curriculum framework. The objective is to teach a number of aul 
nitive skills which are fundamental to creative thinking and then determine wheth® 
such generalized skills can be applied to a specific subject matter area. d 

Before specified learning can take place the learner must seek а" 


. . . D е Ж ыс». 1 g 
accept the learning situation. The probability that an individual will seek or © 
cept a new learning situation is 


Mie , ms 

referred to as motivation by Richard deae, 
and Virginia Carpenter. When a tendency to avoid a learning situation іпсгеа* 
ie., when there is a la 


15 
ck of motivation, the problem becomes crucial. The y^ 
cussion of this problem by deCharms and 


à Carpenter is based on their work V , 
inner-city youngsters. In their chapter "Measuring Motivation in Culturally Dist, 
vantaged School Children" the authors propose a model of motivation in W le 
ранд perceived to be originating his own behavior is called an "Origin" V 
twee to be behaving in a certain way because of an external P'e 
ness, bus => es Чен like a "Pawn." "Pawn behavior implies ae" 
wield ce кайыр implies freedom of choice, not necessarily the ab e 
3 over others." The theoretical framework of the “ОР” model !5 
cussed in detail, 


т мё 

Used in . The n achievement procedures developed by McClelland | chil 

el measuring motivation in a sample of culturally disadvantaged scho? | 
en. The data obtained 1 


à : 55€ 
end to support the theoretical point of view expe 
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Organized Retention and 


Cued Recall 


EVER SINCE Hermann Ebbinghaus 
(1885) brought phenomena of memory under experi- 
mental scrutiny in the psychological laboratory and 
William James (1890) eloquently epitomized the dis- 
tilled wisdom of other great thinkers on the subject, 
research on remembering and forgetting has been 
growing by leaps and bounds. Everyone agrees that 
this research has by now produced an immense quan- 
tity of empirical data, but few are willing to claim 
that it has created any genuine novel conceptual in- 
sights. More often than not, new empiricalfindings 
have generated more heat than light at the theoreti- 
callevel. Thus, despite everything we believe we 
know about the role of a multitude of experimentally 
manipulable variables on verballearning, retention, 
and recall, we seem to be no nearer to the under- 
Standing of the basic nature of me mo ry than were 
psychologists and other writers of previous genera- 
tions. If someone asked us today what do we know 
that is truly worth knowing and that was not known 
to Ebbinghaus and James, we would be hard pressed 
to provide a convincing answer. 


Those who share this pessimistic assessment of 
the state of affairs in research on verbal learning 
ànd memory have been quick to suggest remedies, 
but their advice has been inconsistent. Some have 
argued that experimental conditions between labora- 
tories be more rigidly standardized, others would 
like to see a greater diversity of methods. Some ad- 
Vise that only limited sets of carefully calibrated ma- 
terials be used, others insist on a fuller variety of 
materials, Some are convinced that true understand- 
In£ of memory can be achieved only through mam- 
moth parametric Studies, others are equally certain 


=== Endel Tulving 


University of Toronto 
Toronto, Ontario 


that at least for the time being the solution lies in 
simpler ‘‘miniature’’ experiments. Iam listing 
these conflicting attitudes not to pass judgment on 
them, but rather as a background for my own favor- 
ite complaint about the general orientation ofre- 
search in this broad field. 


My complaint is that far too little attention ha s 
been devoted to the implications of the well-known 
fact that subjects in verbal learning and memory ex- 
periments are not passive organisms. Too many 
students of human learning and memory have predi- 
cated the interpretation of their data and the formu- 
lation of their theoretical statements on the ba s ic 
assumption that in learning and remembering cer- 
tain things happen to the person or cer tain events 
take place in his nervous System, and that the se 
events are governed by nothing more than the present 
and past inputs. Eventually, of Course, science may 
be able to interpret all phenomena of memory with- 
out recourse to a highly Sophisticated, conscious, ac- 
tive, thinking organism, but at the Present time 
nothing much seems to be gained, and a good deal 
may be lost, if we pretend that the articu 
subject is a passive recipient of external Stimula- 
tion, and that acquisition, retention, recall, trans- 
fer, interference, etc., occur in а mental vacuum. 
If we know anything at all about what Subjects do if 
verbal learning and memory experiments, we know 
that they use plans (Miller, Galanter, and Pribram 
1960), that they consciously and deliberately em 155 
strategies (e.g., Pollack, Johnson, and Knaff 1959) 
or mediational devices (e.g., Bugelski 1962: U 
derwood and Schulz, 1960 f ке 


), or simply thatth A 
(e.g., Mandler, 1965). There are some SE 


late human 
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argue that these covert activities of the subject, and 
their behavioral manifestations, merely accompany 
processes of memory and that they play no signifi- 
cant role in determining the quantity and quality of 
a person’s mnemonic performance, but such views 
are rapidly losing their appeal. A great deal of evi- 
dence accumulated in recent years, added to evi- 
dence resurrected from isolated and forgotten ex- 
periments from the past, points to the witting or un- 
witting use of plans, strategies, and mediational de- 
vices by the subject as the sine qua non of remem- 
bering. A fuller understanding of what the subject 
does when faced with the requirements of memory 
tasks of various kinds, therefore, seems indispensa- 
ble for the comprehension of many, if not all, phe- 
nomena of memory. In all probability, an active 
search for such understanding will turn out to be 
more iruitisl than the mere introduction of standard- 
ized conditions of experimentation, calibration oi ma- 
terials, or mammoth experiments. 


In the present paper I will describe some recent 
work we have been doing at Toronto to probe into 
some of the long-standing problems of memory. 
Two questions that I have been particularly inter- 
ested in are, first, why is repetition effective in in- 
creasing the amount of material the subject can re- 
call, and secondly, how does retrieval of mnemonic 
information from the memory storage take place. I 
will obviously not be able to answer these questions, 
but I would like to offer some suggestions relevant 
to them. These suggestions are firmly anchored to 
the basic assumption that the success of retention 
and recall is to a large extent determined by what 
the subject does with the material he isto memorize, 
by the methods he uses to organize the material, 
and by the cues that he uses to effect the recall of 
retained material. The data I will report, I believe, 
will offer good support to this assumption. 


FREE RECALL AND ORGANIZATION 


The experimental data come from experiments in 
which the subject's task is to learn, that is to com- 
mit to memory, a set of discrete verbal items. 
Some of these experiments are concerned with free 
recall of items, while others constitute variations 
on the theme of free recall. In the free-recall ex- 
periment the subject is presented with a list of 
familiar items, such as meaningful words, and his 
task is to remember as many of these items as pos- 
sible, in any order that they occur to him. Thusthe 


3 aradigm constitutes a test for the ac- 
рет апа utilization of nene infor- 
mation about the membership of ortet] as 44 
items in an unfamiliar set. re х 
Eos task, like other paradigms of verbal вава 
and memory, involves mnemonic рек [бл be 
kind often required of individuals outside the «i z 
tory. A teacher’s knowledge of names of stude s 
in a class, a person’s memory forthe na a ad 
guests at a cocktail party he attended the nig 
fore, and a child's recollection of animals he sawon А 
his visit to the zoo are based on retention of informa 
tion of the kind that we investigate in free-recall ex- 


periments. 


I have been particularly interested in free-recall 
phenomena for three reasons. First, the require- 
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ments imposed on the mnemonic information proces- 
sing mechanisms in the free-recall task are neces- 
sarily present in any other paradigm. The subject 
cannot learn the order of a set of items ina serial 
learning task, or specific inter-item associations in 
a paired-associate task, unless he knows what items 
constitute the set, This means that the processes in- 
volved in free recall are in some ways more basic 
than those underlying many other laboratory tasks in 
memory, and that therefore the study of these proc- 
esses constitutes a logical starting point for the 
study of memory. Secondly, Íree-recall phenomena 
were ignored by students of verbal learn ing and 
memory for a long time. | As a consequence they 
have not yet been fitted into a relatively fixed theo- 
retical framework as have been serial- and paired- 
associate learning phenomena. Thus, when we are 
faced with the problem of interpreting experimental 
findings in free recall, we are not hampered by © 
tant theoretical notions. Thirdly, and most impor" 
tantly, it is easier to find out in free-recall situa- 
tions what the subject is doing when he is memoriz- 
ing a set of items. The task set to the subject in 
serial- and paired-associate paradigms is too rigidly 
prescribed by the experimenter and all that can be 
observed at the behavioral level is how well the sub- 
ject can meet the experimenter’s requirements, In 
free recall, the subject is told to recall as many 
items as possible, but he is left free to recall the 
items in any order. Examination of the order in 
which he recalls the items provides considerable іп” 
sight into the subject’s covert activities in the proc” 
ess of memorization. 


In multi-trial free-recall tasks, the same set of 
items is presented on a number of successive trials, 
the order of presented ite ms usually varying fro™ 
trial to trial (cf. Waugh, 1961), and after each input 
trial the subject recalls all items he can. When the 
number of items the subject recalls are pl otted » 
against trials, a typical negatively accelerated lear 
ing curve results (Miller and McGill, 1952; Murdo? 
1960; Tulving, 1962a, 1964; Waugh, 1961). One 9. 
the questions I wish to raise is, why does such learn 
ing, i.e., improvement in trial-to-trial recall, 01 
cur. Ihave argued elsewhere (Tulving, 1962a, 196 
that learning in multi-trial free recall is aconS@” 
quence of the fact that the subject organizes list 
items into larger units of material, into subjective 
units or S-units. At any given time the subject can 
retrieve from memory only a limited number of unit? 
of a particular set of materials, butthe larger thes 
units, that is, the more individual items each um? 
contains, the larger the number of items recalle 
(Miller, 1956a, 1956b). 


ORGANIZATION AND LEARNING 


The original evidence for this contention was pro” 
vided by the observations that the amount of organi” 
zation subjects imposed on the material increased 
systematically over tr ials, accompanying the in 
crease in the number of items recalled, and that 
when the amount of practice was held constant, the? 
was a sizable correlation between recall scores 
organization Scores as subject variables. Such 
relational evidence, of course, is not very compels 
One can justifiably argue that organization andres, 
are two parallel but otherwise unrelated manifes e 
tions of some common underlying mechanism. 


cot” 
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Writers (e, g. 
deed claime 
Pendently of 


‚ Asch and Ebenholtz, 1962) have in- 
d that free-recall learning occurs inde- 
сопаедцепсв of epe. Lo associations, being a 
ing from с Бараг availability of items result 
extent that gei presentation of items. To the 
develipmed c. C eee of S-units reflects the 
denial OPUS of Specific inter- item associations, the 
in свеса de role of inter- itemassociations 
the Spenden E arning also constitutes a denial of 
Ses. How a e learning on organizational proces- 
between es e evaluate the hypothesized relation 
directly he Te oes and free-recall learning more 
Correlati 25 possible on the basis of observed 
lons between the two measures? 


fe aus Possibilitie 
3 to Present a set 
itions whe 


S suggest themselves. The first 
of items to the subject under con- 
items, but = he makes repeated responses to the 
Organize th Оез not actively and deliberately try to 
ing ater ac material into larger S-units. If learn- 
rected re S upon organization, then such non-di- 
ability or penton should have little effect on recall- 
Struct subj ems. The second possibility isto in- 
tion, oe to use effective methods of organiza- 
Structions © hing depends on organization, such in- 
e have q would be expected to facilitate learning. 
One both types of experiments. 


REPITITION AND ORGANIZATION 


At fi 
Tst glance, it may seem difficult to create 


Cond; A 
Saposa under which subjects are repeatedly 
resPunseg à set of items and where they make overt 
thus rgan to these items, but do not think about and 
поо А the items. Since we initially knew 
Start With the the problems involved, we decided to 
must adiu s most obvious approach, although,I 
I ie, Subje at the time we did not have much faith 
Мога ү MES were simply shown a set of common 
1 Шев, апа *d to read these words aloud a number of 
t еге giy ш immediately after such reading, 
ica ев. 1с Same set of words to learn under the 
Ошто] Bu recall instructions (Tulving, 1966). А 
ical c р learned the same set of words under 


Onditions A = Wë 
геһапа but without having seen t 


The у; 
f li. 
thetteney о be learned contained twenty-two high- 
Worg Piects &lish words. In the “reading task,” 
= tin Опе group were shown each of the 
mes, in an uninterrupted sequence of 132 
a he words were exposed one at a time 
сац о e Cond memory drum, atthe rate of one 
Out eac fue the subjects were instructed to 
With ther grou ord as it appeared. The subjects in 
ing а list of P went through the same procedure 
lean” reading (ty уо names. Immediately follow- 
first the same ask, all subjects were asked to 
recai OUP had [ed of twenty-two words to which the 
Senta Nstructio сеп exposed already. Typical free- 
E e is were given, words were pre- 
input 9f one word/second, and at the 
ve изе Subjects had 60 seconds for 
9f wor Thus, bot. Subject received twelve learning 
ferent ds Under ide £roups learned the identical list 
ance С Мав that ntical conditions. The only dif- 
Group ha 


А е Subjects іп the Prior Acquaint- 
St finished reading aloud the same 


в] 


FIGURE 1. Learning curves for two groups of sub- 
jects. One group received six trialsof prior ac- 
quaintance with the set of twenty-two words to be 
learned later, the other group had no such prior ac- 
quaintance, 


20. 


ш / EXPERIMENT 0-14 B 


© NO PRIOR ACQUAINTANCE 
© 6 TRIALS PRIOR ACQUAINTANCE 


MEAN NUMBER OF WORDS RECALLED 


=a er в 901-08 
TRIALS 


words on six continuous reading “trials, ” while the 
subjects in the No Prior Acquaintance Group had 
been **warmed up” with a different set of words. 


The two learning curves f rom this experiment 
are shown in Figure 1. The Prior Acquaintance 
Group had a small, but statistically non-significant 
edge over the No Prior Acquaintance Group on the 
very first trial, but none thereafter. Over all 
twelve trials, the mean number of words recalled 
by the former group was 15. 71, while the mean num- 
ber of words recalled by the latter group was 15. 91. 


These results thus suggest that mere repetition 
of list items had no effect on subsequent memoriza- 
tion of these items. According to the present inter- 
pretation, repetition had no effect because the sub- 
jects did not attempt to organize items in any way. 
Post-experimental interviews revealed indeed that 
very few subjects had thought that they might be 
asked to recall the words they were reading in the 
original reading task. Most of them said they 
thought the experimenter was interested in how well 
they could read, how fast they could read or how 
clearly or consistently they could pronounce the 


words. 


Although we did not give an immediate recall test 
to the subjects at the end of the preliminary reading 
task, it is extremely unlikely that they would not 
have remembered any words that they had seen. 
Thus, this experiment does not provide any evidence 
against incidental learning measured in a single re- 
call test following a reading task of the kind we used, 
It only demonstrates that directed memorization un- 
der the conditions of free recall is not facilitated by 
whatever prior incidental learning has oc curr ed. 
This is not a surprising result, if we assume that 


no organization occurs during the readingtask, that 
incidental memory of items read, therefore, would 

involve individual items as S-units, and that the sub- 
ject can only recall a constant limited number of S- 
units on any given trial in free recall. Ifthe subject, 
on the first trial in the directed learning task,re- 
calls some of the items he learned incidentally, he 

could not recall too many other items because his 
limited retrieval capacity would already be partially 
exhausted. One of the implications of thesefindings 
is that if the subject, before the presentation of 
a longer list for directed recall, were told what some 
of the items in the list are, his recall would be no 
greater than that of the subject who has no such prior 
information, provided the items аге not meaningfully 
related to one another and that the rate of presenta- 
tion is rather fast. The hypotheses that increased 

recall can occur only as a consequence of organiza- 
tion and that the retrieval capacity is limited to a 
fixed number of S-units would completely account for 
this finding, even though it could not be readily pre- 
dicted from many other theoretical points of view. 


ALPHABETICAL ORGANIZATION 


The second type of more direct test of the rela- 
tion between learning and organization, as I said, in- 
volves manipulation of organizing strategies through 
instructions to subjects. An initial study of this kind 
was done as a classroom experiment (Tulving, 1962b) 
The subjects were eighty university students. They 
had to learn a list of twenty-two nouns, each begin- 
ning with a different letter, and the whole set cover- 
ing a wide range of Thorndike- Lorge (1944) frequen- 
cies, In the input phase of each trial, words were 
projected on a screen at the rate of 1. 5 second/word. 
In each output phase of 90 seconds duration subjects 
wrote down as many words as they remembered. 


At the beginning of the experiment, all subjects 
were given typical free-recall instructions. For the 
first three trials all subjects worked under these 
identical instructions. Before the fourthtria 1, 
Subjects were asked to open and read additional in- 


Structions on folded sheets handed out to them before 
the experiment. 


h X For half the subjects, the instruc- 
tions simply reiterated what all subjects had been 
fold at the beginning of the experiment, namely to 
do your very best on each recall trial and put down 
as many words from the list as youcan."' The in- 
Structions for the other half of the subje 
Try to organize your recalled words al 
When you look at the words on the sere! 
first letters, and make an attempt to associate the 
word with the letter, When you write the words 
down, go through the letters of the alphabet one at 
a time and try to remember the word that goes with 
each letter, ”” Subjects in each group, the Standard 
Group and the Alphabetical Group, were given 30 
Е to read these additional instructions. The 
5 i iment was then continued under otherwise iden- 
1cal conditions for eight more trials. 


cts read: 
phabetically. 
en, note their 


The learni 
in Figure s curves for the two groups are shown 


d i i x 
Ше curves ac under identical standard instructions 


differences е almost indistinguishable indicating no 


of the two groups. 
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FIGURE 2. Learning curves for two groups of sub- 
jects. After Trial 3, one group was given instruc- 
tions to recall the words alphabetically, while the 


other group continued learning under standard in - 
structions. 
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From Trial 4 on, however, the curves draw rapidly 
apart, the mean number of words recalled by pike 
Alphabetical Group on Trial 5 equaling that of us 
Standard Group on the last trial of the experiment 
Trialll. Judging by the shape of the curve of t a 
Standard Group, it appears that it would have e T" 
them many additional trials to reachthe level of P a" 
formance comparable to that reached by the AE 
betical Group on Trial 6. The differences betwee 
the two groups would undoubtedly be even greaten i 
it were not for the artificial limit on the porion i 


ance of the Alphabetical subjects impos ed by 
“ceiling effect. ” 


d 

These results clearly indicate that meth? ing th 

ganization are very important indeter Mİ” 
course of memorization of a list of items- 

betical organization is apparently much more 


tive in this type of task ue 
thai erme 
Organization sinh 


~ grouping of words by their meaning 
used by the subjects in the Standard Group. v4 
learning curve of the subjects in the Standard Gro i$ 
reflects their limited performance, but the limit i 
not wholly determined by their inadequate mne mon 
ability. То а large extent, it is determined by the? 
ignorance as to how to make best use of their mne 
monic ability in this type of task. When subj есі? 
аге given more powerful organizing instructi? 2 
their ‘‘ability”’ increases considerably. 


ес" 
е a 
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I аз E imports implicationof this observation — 
theme = oing to digre s somewhat from my main 
еласа n Lm do with the problem of individual dif- 
study mAN "ies ning ability. It is commonplace to 
dividing s b ual differences in learning ability by 
on the de )Jects into “fast” and **slow"' learners 
and then E their performance inalearning task, 
био grou 20] for similar differences between the 
ious ps eos in various other situations, such as var- 
Send. tests. Inthe experiment I have just 
memorizati lere were pronounced differences in the 
two gross D performance between the subjects in 
domly it ae pus the groups were selected ran- 
have Shown а foregone conclusion that they would 
frequent} no differences оп any other tests. How 
"ability dete individual differences in learning 
manipulated. ег mined by the same factors as those 
istory of ¢ in the present experiment? The long 
Telates of үнген to find powerful psychometric cor- 
lations Bees hing performance, or sizeable corre- 
at Individual. different learning tasks, suggests 
2e largely val differences in learning “ability” may 
€Ctive ЕШЬ by individuals' knowledge of ef- 
er than by pees of handling a particular task, rath- 
any rate in theme goodness of their memory. At 
Seems that if is light of data from this experiment it 
individual аА wish to add to our understanding of 
Place to st ‘ferences in learning “ашу,” a good 
Memorinie "S ie analyze the learners' strategies 
o Observe 116 in a given learning situation, and not 
ental sit is behavior in some other extra-experi- 
uation, 
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. Le 
ization 2 now return to the relation between organ- 
ject na 7d learning, We have seen that when a sub- 
Words а memorize а list of randomly selected 
than othe Ptica] organization is more effective 
ization one odg of organization. Alphabeticalor- 
i saia nges the task into a kind of paired as- 
No Serv, ning task, The initial letter of each 
е ре °З as a cue for the recall of the whole word 
Cues ines enter does not have to present these 
ET ES an articulate subject can generate the m 
in tion each We could say that in alphabetical organ- 
qtia 1 ter Put word is cued with respect to its 
he Avai]ap;;, "d that its retrieval is facilitated by 
$08 g or ility of relevant cues. When other meth- 
items i terization are used, such as grouping of 
ro longingne ms of their associative meaning, their 
wks or le Into superordinate conceptual catego- 
ords are d egree of their prior familiarity, input 
su ап the fad With respect to their properties oth- 
Si y effe itial letter and their retrieval is pr e- 
ible, о Cted through these other cues. It is pos- 


» Ое 
eral Ways Es that a given word is cued in sev- 
Ourse or ihe that the way it is cued changes in the 
Press experiment, but under the usual time 


re i à : 
Probably a а typical experiment, multiple cueing is 
exception rather than a rule. 


Suppo, 

ei of ln on that a subject has been exposed to a 
а уеге alph: а потре of trials, that he has not 
o he has Bent etical method of organization, and 
sunt Character; 46106 the words in terms of their 
Ubject ere €ristics. What would happen, if the 
TBanization ” told to switch to another method 
» Say alphabetical organization? We 


ао 


know already that alphabetical organization is likely 
to be more effective than other methods of organiza- 
tion and that therefore alerting the subject to it 
should facilitate learning. But it is also possible 
that changing the method of organization requires not 
only the adoption of the new strategy, but also the 
suppression of the previously used strategy. One 
might expect, therefore, that immediately after the 
introduction of alphabetical instructions the effi- 
ciency of the storing and retrieval mechanisms 
is temporarily reduced and that the disturbance is 
the greater the more highly developedthe non-alpha- 
betical organization. 


Data relevant to these hypotheses come from an 
experiment done by Marcia Ozieras partof her PhD 
thesis (Ozier, 1965). Her subjects learneda list of 
twenty-two words, each beginning with a different 
letter. Words were projected on a screen at the 
rate of 1. 5 second/wordand subjects recorded their 
recall on paper-pullers that prevented the m from 
seeing more than one word of their recall at any 
time. The duration of output phase of eachtrialwas 


90 seconds. 


Most subjects were initially given standard free- 
recall instructions and then switched over to alpha- 
betical instructions at some point in the experimen- 
talsequence. The two independent variables in the 
experiment were, first, the ordinal number of the 
"critical" trial on which alphabetical instructions 
were given, and second, pre-input versus post-in- 
put alphabetical instructions, Different groups of 
subjects were given alphabetical instructions on Tri- 
als 1, 5, 9, 16, and 21. In each of these groups, 
half the subjects were given alphabetical instruc- 
tions before the input phase of the critical trial, 
while the other half received these instructions after 
the input phase, that is, before the output phase, of 
the critical trial. Before the critical trial, subjects 
in a given group worked under standard free-recall 
instructions. Thus, for instance, one group of sub- 
jects worked under free-recall instructions for eight 
trials, then received alphabetical instructions, fol- 
lowed by the input and output phases ofthe ninth and 
subsequent trials. Another group worked under 
standard free-recall instructions for eight trials, 
was then shown the words in the input phase of the 
ninth trial, and then was asked to start recalling 
words in the alphabetical order, beginning with the 
immediately following output phase. There was thus 
a total of ten independent groups, with twelve sub- 
jects per group. Scoring was lenient; even if the 
subject had deviated somewhat from the alphabetical 
order when he was supposed to recall the words al- 
phabetically, he was given credit for each wordcor- 
rectly recalled. Subjects who obviously did not fol- 
low alphabetical instructions, however, were re- 


placed. 


The somewhat schematized results of this exper- 
iment are shown in Figure 3. The upper he avy 
smoothed curve shows the learning curve for the 
subjects who received alphabetical instructions be- 
fore Trial 1, the lower heavy smoothed curve rep- 
resents the free-recall conditions. Sub jects Wi 
had received pre-input alphabetical instructions o x 
Trial 1 had a mean recall of approximately ej Wt 
words on that trial, while those who had саана 


FIGURE 3. The effect of alphabetical organizing 
instructions given at various points inthe course of 
learning. See text for explanation. 
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post-input alphabetical instructions on Trial 1 re- 
called approximately five words, a difference of 
three words. The learning curvefor the latter group 
is not shown on the graph. It reached the learning 
curve of the former group by the fifth trial, being 
practically identical with it thereafter. 


The other data in Figure 3 show the initial drop 
and subsequent rapid recovery in recall of groups 
who were given alphabetical instructions at Trials 
5, 9, 16, and 21. For each of these groups, with 
the exception of those for whom Trial 21 was the 
Critical trial, mean recall scores are shownfor five 
trials after the critical trial. These data show Г 
first, that immediately after the introduction of al- 
phabetical instructions there is a Ягор in perform- 
ance - with the exception of pre-input instructions 
on Trial 5 — with the magnitude of the dro: 
directly With the ordinal number of the bert dere 
and being Consistently greater for post- input alpha- 


betical instructions, and se 

Я cond, thatthe subsequent 
recovery of the effectiveness of al habeti i e 
tions given on the criti Я соо 


о › caltrialvaries inversel 
with the ordinal number of the Critical trial. ý 


The percent: 
phase followin 
structions is plotted as afunction of the ordinal num- 
ber of the critical trial, for both pre- input and post- 
input instructions, in Figure 4. Each data point in 
Ha ead represents the mean difference in recall 
a E number of words recalled onthe critical 

pee е immediately preceding trial divided by 
ааа of words recalledonthe immediate 
Кой. a trial, The data plotted in F igure 3 
Suetion 3. ne in performance after the intro- 
test co phabetical instructions, measured in 
siihe ше number of items, varied directly 
rm сш inal number of the criticaltrial. The 
n in Figure 4 indicate that the Same rela- 
ion holds between percent: 
ordinal! number 


age loss in recali and 
of the critical trial, andthe 


age loss in recall in the first output 
g the introduction of alphabetical in- 
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From previous experiments we already know that 
organization of material increases over trialsof 
practice in the free-recall situation (Bousfield, Puff, 
and Cowan, 1964; Ehrlich, 1965; Tulving, 1962a, 
1964). Therefore, manipulating the tempora! point 
at which subjects are switched from other forms of 
organization to alphabetical organization e ntially 
amounts to manipulation of the degree of existing OF" 
ganization that has to be changed. The results of 
Ozier's experiment suggest that the stronger the 
pre-existing organization, that is, the more firmly 
the list- items are cued with respect to characteris- 
tics of words other than their initial letters, the 
more disruptive is the requirement that the subject 
change his system of cueing and the longer it takes 
for the otherwise powerful alphabetical sy st e m of 
organization to become effective. 


AVAILABILITY AND ACCESSIBILITY 
OF STORED INFORMATION 


I have so far talked about two general classes of 
methods of organization: organization of words int? 
larger groups in terms of their perceived, discov” 
ered, or fabricated meaningful relations to one an^ 
other; and organization in ter ms of structur al 
features of words, such as initial letters. In eithe” 
case it is as if words stored in memory - or infor 
mation about words to be reproduced - are in stor” 
age in a way that would facilitate retrieval: two 07 
more related words are stored in or near the sam 
location, suchthat when the retrieval mechanis? 
finds one, the other one is also easily located, a 
else a word is stored in a location in which its initia" 
letter is stored. Thus, one part of the stored ал 
formation can be thought of as a retrieval cue for“, 
other part. Organization of information in the S 
age, however, can facilitate retrieval only to шее 
tent that retrieval cues are available at the time ive 
recall. Alphabetical organization is more effect jc 
than organization of items interms of their seman 


FIGURE 4. Percentage loss of recallon the tri 
immediately following the introduction of alphab 
ical organizing instructions, as a function of the 


Hen 8 
“critical” trial on which alphabetical instructio" 
were given. 
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f 

Aus because the initial letters of the words, 

es ing as retrieval cues are readily available to the 
Ject at the time of recall. 


ar [йв kind of speculation implies that organization 
aterial in the memory storage, or organized 
салга does not necessarily increase the storage 
tips o but rather makes the stored information 
е йер: to recall by providing distinctive 
formar Cues through which individual items of in- 
аб: a can be reached. It also implies that the 
Stóra e information potentially available in the 
ord ata given time may be greater than is ap- 
аан typical unaided recall. In other words, 
limited on of memory is primarily determined by the 
necessario у of the retrieval mechanism and not 
mechanisn., by the limited capacity of the storage 


So 11 experiments of the kind that we have examined 
tion and is impossible to distinguish between reten- 
Eanizati recall. If a subjectusing alphabetical or- 
compara! can recall more words than an otherwise 
we do Б le subject who uses less efficient methods, 
ble to diti pis whether this difference is attributa- 
is аең ы in retention or only in recall. It 
number e that both subjects have retained the same 
Simpl: a words and that the alphabetical organizer 
worda. bu access to a larger number of retained 
Organiz "t it is also possible that the alphabetical 
mation €r in fact has been able to store more infor- 
ion ig in his memory, that therefore more informa- 
or negate in the storage, and that his superi- 
formation re reflects the greater amount of in- 
Оп that he has stored. 


ration’ logical requirements for the analytical sepa- 
een ауз. tention and recall, or the distinction be- 
rial, үш ШАР and accessibility of stored mate- 
Present t Simple enough. All that we need todo is to 
Subjects he material to be remembered to different 
Heal IgE identical conditions, including iden- 
number р. tions, and then vary experimentally the 
e ied retrieval cues at the ti me of recall. If 
e n are treated identically upto the beginning 
information test, we can assume that the amount of 
information they have stored and thus the amount of 
the pd available is the same for all groups. If 
Е = and kinds of retrieval cues given to dif- 
poces in ups at the time of recall produce differ- 
es pocan, these differences would reflect dif- 
and not ditt accessibility of the stored information 
ferences in availability. 


In the in 


een itial experiment we did to distinguish be- 


Pear lstor bility and accessibility (Tulving and 
Consistin » 1966), we used a dual-level input list, 
RS instanc, of category names and appropriate words 
One of t n Of these categories. Thus,for instance, 
velve ee consisted of twenty -four words in 
list Present 165; with two words per category. The 
this; thin €d to subjects might contain items like 
Substancen paund on a farm - WHEAT, TRACTOR; 
Ekologia Пони [ond - SUGAR, CLOVES; 
tuments — ions - PLAIN, LAKE; musical in- 
Structed d . ORUM, FLUTE; etc. Subjects were in- 
Words ав at their task was to remember as many 
Possible, that they would be tested for the 


words only, and that they did not have to memorize 
the category names. All subjects learning a partic- 
ular list were treated identically up to the beginning 
of the recall test. At that point, half the subjects 
were asked to recall all the words they remembered, 
i.e, words other than the category names, while 
the other half were presented with the category 
names as retrieval cues and asked to write down all 
the words belonging to each category that they re- 
membered from the list. 


The design of the experiment included lists of 
three different lengths (twelve, twenty-four, and 
forty-eight words) and three levels of numbers of 
words per category (one, two, and four). Within 
each of these nine conditions of presentation, there 
were then two conditions of recall, cued and non- 
cued, as I have just described. The experiment was 
done with high-school students of both sexes as sub- 
jects, and data were collected in regular class ses- 
sions. Input material (category names and words 
to be remembered) was presented by means of tape 
recorder. The subjects wrotetheir recall in appro- 
priately designed booklets. The presentation time 
varied with the length of the listand with the number 
of categories according to the formula: T- NoC«L, 
where T is presentation time in seconds, NoC isthe 
number of categories, and Lis list length. The 
amount of time given for recall was 1, 2, and 4 min- 
utes for lists of twelve, twenty-four, andforty-eight 
words, respectively. A total of 949 subjects was 
tested, with approximately fifty-three subjects for 
each of the eighteen conditions of the experiment. 


The mean number of words correctly recalled 
under each of the conditions of the experimentis 
shown in Table 1, together with corresponding 
standard deviations. As these data show, cued re- 
call was superior to non-cued recall for all nine 
lists, the superiority varying directly withlist 
length and inversely with the number of words per 
category. The difference between cued and non-cued 
recall thus was smallest for the list in which twelve 
words were organized into three categories, with 
four words per category. In this case, presentation 
of category names as retrievalcues apparently does 
not help subjects very much since they presumably 
can remember the category names on their own, 
The largest difference was found for the 48- word 
list in which each word belonged to a different cate- 
gory. Presentation of category names as retrieval 
cues in this list facilitated recall considerably: sub- 
jects in the cued recall group recalled over thirty- 
five words on the average, while the subjects in the 
non-cued group recalled less than sixteen words. 


The major implication of the results of this ex- 
periment is that the subject's failure to recall a 
word he was instructed to recall does not necessar- 
ily mean that information about the membership of 
the word in the list is no more available inthe mem- 
ory storage. It is available, but not accessible un- 
der the typical unaided recall conditions, If the sub- 
ject receives cues about the **location'' of the rele- 
vant information in the storage, he will “fing” e 
otherwise inaccessible words. Thus, intratrial vat 
getting" (Tulving, 1964) of at least some of the it or- 
does not reflect the failure of the storage mecha 
but only the failure of the retrieval mechanis ism, 
does the retrieval mechanism operate? ви. Ноз 
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TABLE 1 
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MEAN NUMBER OF WORDS RECALLED AND THEIR STANDARD DEVIATIONS FOR THE EIGHTEEN 


EXPERIMENTAL CONDITIONS 


Number of Words Per Category 


List Recall 1 2 3 

engm Conditions M S. D M S. D. M S. D. 
Cued 10. 70 1.57 10. 94 1.12 9. 98 1.55 

" Non-cued 7.10 1. 61 8.13 2.03 9.31 1.91 
Cued 21. 70 1.97 19.31 2.38 15. 11 3.17 

" Non-cued 11.18 2.56 11.82 2.54 13. 38 3. 08 
Cued 35, 35 6.42 35.76 7.86 29. 60 6. 00 

Е Non-cued 15. 57 3. 53 18.79 5. 99 19. 33 5.34 


The Tulving and Pearlstone (1966) experiment рго- 
vides some data relevanttothis question. Word-re 
call data in an experiment of this type, where words 
are organized into higher-order units by the experi- 
menter — in this case, into superordinate cate go- 
ries — and where this organization is adopted by the 
subject, can be analyzed into two independent multi- 
plicative components, number of categories recalled 
and number of words recalled per recalled category 
(cf. Cohen, 1963, 1966). A category is said to be 


recalled - to be accessible — if at least one word 
Írom that category is recalled, Given the number 
of tot 


al words recalled and the number of categories 
recalled, the mean number of words recalled within 
recalled categories can be obtained simply by divid- 
ing the former number by the latter. For instance, 
if the subjec 


t recalls the words SUGAR, CLOVES, 
gories - sub- 


two words from о 


per category, 
other, 


ne category, and one from the 


or the 24-word and 


number of words recalled per c. 


ant ategory was invari- 


These data su 
S ggest then that recall of words f 
accessible categories is independent of the nominen 


иа 


of categories recalled, regardless of whether differ? 
ences in category recall are brought about through 
presentation of category names as retrieval cues OF 
through increasing the list length and thus the total. 
number of categories presented. Retrieval mecha" 
ism seems to operate in Some sense independently 
at two levels: at the level of higher-order units of Р 
organization and at the level of individualitems with 
in these higher-order units. Limitation of recall 1$ 
determined by limitation of retrieval at both level? 


A further demonstration of the relation betwee? 
number of categories and recall is provided by es 
other experiment of Ozier's (1965). In this expe? 
ment subjects learned lists of twenty-four wor n 
either under conditions of free recall or under ot 
ditions of alphabetical organization. For ашеге 
groups of subjects the number of unique initial 1 ир, 
ters of words in the list was varied, Foronegro er 
all words began with different letters. For anoth ie 
group there were twelve pairsof words (words we 
in each pair beginning with the same letter); E 
third group, eight initial letters were used, © 


Pi t 
associated with three different words, and во 9^ 
the group in which twelve words began with one ex^ 
ter and twelve wor 


: ds with another. Thus; à Tul 
periment can be thought of as an analogy tO th 

ving and Pearlstone experiment, with list len? of 

held constant at twenty-four words, the numb?" е 
categories (initial letters of words) varying OV°" of 
range of two to twenty-four, and the numb? 
words ber category (words sharing the initial le i 
varying correspondingly over a range of twelv ing 
one. Ozier’s experiment differed from the TUVIE 
and Pearlstone experiment in that it was a mU. o, 
trial learning experiment. Each subject was 8 d 
twenty trials of practice on the list to which he ized 
been assigned. The order of words was random? e 

on all trials, the words were projected on a pers 95 
at the rate of one word/ second, and subjects ha 
Seconds for wr 


ne 
itten recall on each trial. Under t | 


ег) 
o 


П 


"sd 
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аа. Меап recall on successive blocks of 
initial fies 9 lists varying in number oí different 
ine pai 187 ѕ and learned under alphabetical organ- 
ferret (A), and for a list learned under 
liste E , nstruetions (FR). In the designation of 
Тёре the first figure refers to the number of dif- 
no M e of words, and the second rep- 
s ə By UA бшк к) i 
Өй lette umber of words beginning with a com- 


24 
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© 
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o 8r 
Q 
5 
= I6} 
ш. 
o 
a 
ш lak 
3 
z 
12 F 
[е 
че | 1 ' ШИШЕ Jj 
2 3 a 5 
BLOCKS OF FOUR TRIALS 
Condit; 
the V ‘Ons of alphabetical recall, initial letters of 
Phase, 3 were given as retrieval cues in the output 
men ubjects were premedical students, mostly 


» Wi 
Ih twelve subjects per group. 


O 3 
ferenco first trial, there were no systematic dif- 
ent Broups. the number of words recalled by differ- 
ferences and only after the first three trials did 
е Presenta rome noticeable. This suggests that 
T Words lon rate of 1 second/word was too fast 
ters, to be cued with respect to their initial let- 
Eroups ang learning curves for four alphabetical 
5 i n à [ree-recall group are shown in Figure 
Figure 5 ean number of words recalled is plotted in 
logar ig, 45108 five blocks of four trials each on a 
varied direct vy These curves show that learning 
er о uni y with the number of categories (num- 
Umber к Initial letters) or inversely with the 
116 to not Words per category. It is also interest- 
Phabetiea] that free recall is more efficient than al- 
o catego recall of the list in which there are only 
ries and twelve words per category- 
Thi 
that indo elta of the last two experiments suggest 
witi efficient tions where the subject is provided 
generates retrieval cues — or where he can eas- 
the inter- jt them from his own memory- and where 
E associative strength of items within 
S minimal, the optimum strategy is to 


a 
orga Rory 
nize the į 
ei i : S 
tems into a maximum number of cate 
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gories. On the other hand, under conditions where 
the Subject can benefit from organization of individ- 
ual items into higher-order units only to the extent 
that he himself remembers what these higher-order 
units are, the optimum strategy is to organize items 
into relatively few categories. It is most likely, how- 
ever, that when the number of categories is de- 
creased beyond a critical limit the difficulty of re- 
trieval of items from withina given category will 
outweigh the facilitating effects of smaller nuraber 
of categories, and recall will decrease. In the lim- 
iting case where all items in a list belong toa single 
category, recall is no better than recall of a list in 
which each item belongs to a different category. 


SUMMARY 


I have argued that understanding of many phenom- 
ena of verbal learning and memory would be en- 
hanced if we adopted the basic orienting attitude that 
a person's mnemonic performance depends to a very 
large extent on the methods that he wittingly or un- 
wittingly uses to organize the material. Theimpor- 
tant function of such organization is to facilitate re- 
trieval of infor mation from the memory storage. 


I considered in some detail the problemof the ef- 
fect of repetition on recall of a set of discrete items. 
Mere mechanical repetition of items does not seem 
to have any noticeable effect on the subsequent 
directed memorization of these items, provided that 
individual items as such are already well known to 
the subject before the experiment. Learning of the 
set, however, can be considerably speeded upif the 
subject uses effective methods of organizing the ma- 
terial. Individual differences among subjects in 
various learning tasks may reflect primarily their 
knowledge of, and the ability to utilize effective 
strategies and mediational devices, and only second- 
arily their innate differences in memorizing ability. 


Different methods of organizing material may 
be incompatible with one another. When the subject 
has been using one method and then is asked to a p- 
ply some other method to the material he is memo- 
rizing, the adjustment to the new method requires 
some time. Interference with overt recall under 
these conditions reflects incompatibility of different 
types of organization, and adds further support to 
the hypothesis that learning is critically dependent 
upon how the material to be remembered is organ- 


ized. 


It is not necessary to assume that organization 
changes the amount of material that the subject can 
retain, or the amount of information potential ly 
available in the memory storage. Evidence sug- 
gests that organization only serves to make more 
of the available information accessible. The learn- 
er has always more mnemonic information av aj] - 
able, relevant to the successful accomplishment of 
a memory task, than he can ordinarily make acces- 
sible to recall. Accessibility of infor mation di 
pends not only on its availability in the Storage, b A 
also on retrieval cues. Retrieval cues may be A 
of the environmental conditions pr ev a į ling ¢ Fa 
time of recall, or they may have to be qi m 
memory. Limitation of the retrieval mechanism 


p—————————————— 
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thus seems to be determined by the lack of accessi- 
pility of efficient retrieval cues, and the functional 
significance of organization lies in its role of pro- 
viding such retrieval cues. 


Optimum organization depends on the require- 
ments of the recall task. If retrieval cues have to 
be generated from memory, the materialisbest or- 
ganized into a relatively small number of catego- 
ries - Mandler (1967) suggests that this number 
for common words is approximately five – if, on the 
other hand they will be available in the perceptual 
environment at the time of recall, the number of 
categories should be as large as possible. Retriev- 
al cues such as category names provide accessto 
categories or other kinds of higher-order units of 
material, but once access to such higher-order units 
becomes possible, retrieval of individual elements 
within units is unaffected by the number of higher- 
order units already recalled or yet to be recalled. 


The evidence on which this account has been based 
is admittedly rather fragmentary, but the initial re- 
sults are quite encouraging. Certain consistencies 
have already become apparent and further work will 
probably clarify those aspects of the conceptual 
framework that are still obscure. 


While all this research that I have discussed was 
not undertaken with a view to solving practical prob- 
lems as they occur in learning andre memberi ng 
in real-life educational settings, it does have some 
implications for these problems. Iwill review just 


one, fully realizing that it is simply a reiteration of 
what many people have said before. 


The kinds of findings from the sim ple experi- 
ments I have described in this paperonce moreun- 


derscore the need for the examination of me thods 
that learners in classr 


ooms use when they acquire 
knowledge and skills. We can never be certain that 
the methods of learning a person uses are the opti- 
mum ones for the requirements of any giventask 
or that these methods are sufficiently flexible to fit 
Fr ira bep fact do require different str ate- 
Д nd at the i 
нага азон time, we probably could 


material on his Own even if it i 
en if it is not opti 2 
tured by the external environment. тышан 


To the extent th: 
at m 
ods of learning can be 5 
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uations, can do better if he has a good teacher to or- 
ganize the material for him, but he cannot always 
expect to have good teachers around him. In many 
situations he has to be his own teacher. Would it 
be too rash to suggest that instruction in how to learn 
and how to remember be as important a part in the 
curriculum of the schools of tomorrow as аге the 
three R's now? 


FOOTNOTE 


Research reported in this paper has been вир" 
ported by Grant No. APT-39 from the Nation- 
al Research Council of Canada, and Grant No. 
GB-3710 fromthe National Science Foundation. 
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Developmental Processes 


in Thought 


WHEN Professor Klausmeierchose 
the title for my paper, he asked me todeal with 
three points. First, give an outline of the concep- 
tual framework that I am going to use to indicate de- 
velopmental processes in thought. Second, give 
Some account of the research that has been carried 
out to test the validity of the generalframework. 
Third, point out the educational implications of what 
is dealt with in the first part of the paper. 


The treatment of intelle 
ing thought American psychol- 
rSwiss psychology. 


» for Piaget has great in- 
nal circles, although he 

among British psychol- 
Ogists. 


FIRST 21 MONTHS OF CHILD'S LIFE 
_ There i 


- i s life are of great 
irom the poi iew of the growth of 

thought, Piaget hag shown that a baby’s movements 

are not random, indeed, on close inspection they are 

do ра This period of devel- 
e iaget the i i- 
y g period of sensor 


Kenneth Lovell 
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is the 
motor intelligence, because the schemas, that Es 
Sequence and the structure of theactions S ires 
built up in the mind, are dependent upon the prom 
Support of perception and motor activity. „Һар“ 
12 to 18 months of age the child goes throu B n ob^ 
riod of intense experimentation. He will let ae 
ject fall to the floor to see what happens; he xil i 
knock things, shake them, or throw them to S ard 
they roll. New ways of doing things are Варта 
based оп а grasp of new relationships. For ene tall 
a child will pull a box toward him in order to 9 $ 


о 
а toy placed on it, which otherwise would Pc he 
reach. And between 18 and 21 months of SE IO d 
awareness of relationships is sufficiently dev to 


n 
for the child to be able to invent new means, 2P on 


whit 
be able to foresee which acts will succeed and Whig 
will not, 


vention c 
to intern: 


without putting them to the test. Thi nt 
omes through a covert process that aP nd 
al experimentation. The earlier tria ё pild 
error period is not now so important, for the 1008 
сап increasingly represent to himself the VET , 1e 
Possible actions and how they must be combi" y ink, 
attain a desired e 


iani ОР" 
nd. The child is beginning '9 (oy 
for his actions are now carried out in an inwa ge ac 
Some psychologists would call such internali 


tions, incipient or implicit responses. 


di^ 

n or e 

During the first 21 months or soof life, ut р 

nary child elaborates the basic schemas of rur 
ject, of space, of time, and of causality. F 


an 
ample, the basic Schemas in relation to space nis 
time are laid down 


reaching actions fo 
when he moves to 


ex” 


as when the child adjus t ^ n 
r near and distant objects. 
catch a swinging rattle. 
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tø ee bia . the child in adapting himself 
OON nancy ment has to assimilate or absorb new 
time: he vast neo tts existing schemas. Atthe same 
through th 2e accommodate to the environment 
build m "s modification of existing Schemas or the 
Dew situati new. ings. Cognitive adaptation to some 
nehon i оп, and hence the ability to deal with it 
паа P ANNOIVES both these processes. To as- 
must dud meaning of a new situation the child 
date to it poate to it; and in order to accommo- 
tive e ree be able to assimilate it. Cogni- 
new always E thus a step by step process with the 
Perience is building on the old. ) But once new ex- 
Come ease assimilated, the child's schemas be- 
plex Rd ОШ and because of this, more com- 
Child's m ations are possible. Moreover, the 
absence of TBS do not remain unchanged even in the 
nvironmental stimulation, for meanings 


m 
view ОЁ. This internal renovation is, in Piaget's 
Even, so important source of cognitive development. 
note that К ену» growth is slow. We must also 
at it is s eee 5 observations lead him to believe 
Children кч mas in the processof organization that 
Pleasure È to repeat playfully and with seeming 
Banizeq ‘th ut when such schemas have become or- 
Cease to be арракен pleasure disappears and they 
form new repeated unless they are combined to 
avel] (ea or serve as a means to some end. 
and one of ) remarks that schemasare structures, 
that of r their important, built in properties, is 
In the бе ed assimilation of anything assimilable 
vironment. 


BE 
TWEEN 2 AND 8 YEARS OF AGE 


Thr 
шпар quen out the sensori- motor period the child is 
Sent to hi use an image or word, which could repre- 
F ia man object or event not actually present. 
What he Bet, the child moves out of this stage into 
When the alls the pre-operational stage of thought 
Stands child can differentiate a word from what it 
Stang for? for example, use the sound ‘‘dog’ Чо 
he can re; Ог represent, а dog. When he can do this 
longer in Present to himself situations that are no 
entirely actual evidence, and thought is lifted toan 
thought aon level. Note carefully, however, that 
omes before language; the latter is fitted on 
П use re at already exists. But once the child 
Mensely i Buage, thought is extended over an im- 
More per Ncreased range. Moreover, he now has a 
Cutside won N and far more flexible model of the 
ia zi d, for he is no longer dependent u pon 
етее se ception and motor activity for thought. 
the ing tends between 2 and 4 or 5 years of age. 
CS Situation. 10 Center on one striking feature of 
e to Ds is also irreversible in that the child 
chat irom WES back, in his mind, to the starting 
hild а thi ich his immediate thinking began. The 
00) iew 15 stage can only see an event fÉrom his 
9f other d he is unable to conceive the viewpoint 
вида аз ie dren. Yet by 3 ог 4 years of age the 
this а c pled the adult model of language, and at 
*S level of ihe in grave danger of overestimating 
" hought from the nature of his speech. 
y ab 
by Tus d years of age a change begins to set in, 
es more wae of age the child's thinking be- 
™ to cert matized; that is, his thoughts now 
ain rules and his thinking becomes 
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what we adults call logical. The sequence and struc- 
ture of actions in the mind, or the schemas, now 
available are now altogether different in kind. The 
ability to reason and ‘‘understand’’ demands higher 
order schemas which permit a simultaneous grasp 
of the successive sequences of actions taking place 
in the mind. The ordinary child of 7 to 8 years can, 
as it were, ‘‘look down on his schemas" or ‘‘turn 
round on his schemas. " He is then aware of the se- 
quences of action in his mind; he can see the part 
played by himself in ordering his experiences; and 
for any action in his mind, he can seethat there are 
other actions that give the same result. That is, he 
sees equivalences. Thus he understands that 4-2- 
3«3-7- lor that all girls plus all boys plus all 
adults = all boys plus all adults plus allgirls. In- 
deed, there is now anew kind of coordination of 
schemas, yielding a simulataneous understanding of 
equivalences for actions within the mind. Thus the 
child can now measure the same distance infeet and 
inches and understand that the different figures 
mean the same thing; he can perform subtraction us- 
ing the method of complementary addition; he can 
decompose 42 units in 4 tens and 2 units with under- 
standing. Thinking now conforms to a system, and 
there is learning with a certainamount of understand- 


ing. 


At the level of sensori- motor and pre-operational 
thought there was a great deal of learning, but little 
or no understanding in the sense that the child 
learned a linear sequence of actions but was unable 
to elaborate a set of equivalences. We must not in 
any way belittle learning that takes place before 
thought becomes systematic, for such learningis es- 
sential for the growth of thought itself. Indeed in 
older normal children, and in adults, a great deal 
of learning is of this type. 


Now because the elementary-school child can in- 
creasingly see the part played by himself in order- 
ing his experiences, and because he can increasing- 
ly coordinate actions in his mind and make them con- 
form to a system, he can build the concepts ofa 
class, a series, length, time, and soon. But the 
concepts that he builds are on ly those that he 
can attain from experience with first hand reality, 
and his systematized or logical thought is related 
only to the world of perceivable things and events. 
For Piaget, the elementary-school child is at the 
stage of concrete operational thought and is, for ex- 
ample, able to understand certain aspects of mathe- 


matics. 
END OF ELEMENTARY SCHOOL PERIOD 


Toward the end of the elementary-school period 
however, children begin to realize that there аже 
gaps and uncertainties in their thinking, and that cer- 
tain kinds of problems cannot be solved by them, As 
the child becomes better at organizing and structur- 
ing problem data with concrete operational methods 
he becomes more aware that the latter does not yield 
a logically exhaustive solution to his problem Hi 
gropes for new methods of attack, often in Piaget e 
view, as the adolescent commits himself to real life 


situations. 


So from 11 to 12 years of age in able childr 
and from 13 to 14 in ordinary pupils, new tie n, 
J ing 
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skills begin to emerge. Due to his continued inter- 
actions with the cultural milieu, and to the matura- 
tion of the central nervous system, the individual 
can produce more complex expectations when faced 
with certain kinds of situation and data. His logical 
thought is no longer restricted as before, for he can 
increasingly use statements or propositions relevant 
to objects and their relations. The sequence and 
Structure of his mental actions, or his schemas, 
are new in kind. The pupil can now begin to manip- 
ulate statements which refer to classes and relations. 
For example, he can answer the question “И Henry 
is taller than Mary and shorter than Debbie, who is 
the tallest?’ Or he can argue from the situation in 
front of him that, ‘‘Either this must be true or that 
must be true, "or he can see that “X implies Y." 
The pupil can, for any sequence of actions in his 
mind, see a greater range of equivalent actions than 
was the case at elementary-schoollevel. Faced 
with certain situations the child can, at 13 or 14, set 
up an hypothesis and work out what would happen if 
this hypothesis was true. He is now able to deal 
with the merely possible. The pupil has reached 
what Piaget calls the stage of for mal operational 
thought, and many experiments to illustrate the 
growth of such thought are to be found in Inhelder 
and Piaget’s (1958) book The Growth of Logical 
Thinking from Childhood to Adolescence (1958). А 
number 01 these experiments were repeated by Lov- 
ell (1961) and Jackson (1965), and the broad stages 
in the growth of logical thinking were mainly con- 
firmed although the position is more complicated 
than Inhelder and Piaget suggested. It must be 
stressed, however, that formal thought is found in 
all areas of thinking — in history, literature, poli- 
tics ~ and not just in science and technology. There 
is much published work to Support this statement, 
and if space permitted, I could give you some inter- 


esting illustrations for formalthink ing inthese 
fields. 


With the onset of formal thought, the pupilis able 
to elaborate an entirely new kind of concept. You 
will remember that in the elementary school the 
child could say how he ordered his experience, for 
he could dissociate the part played by himself in 
classifying his experience from the Characteristics 
of that experience. But in adolescence the pupil is 
able to structure and coordinate actions upon rela- 
tions, which themselves result from the coordina- 
tion of actions. For example, in the case of heat in 
physies, the concept depends on the earlier elabor- 
ation of the concepts of mass and temperature, The 
latter are developed at the level of concrete opera- 
tional thi some 


tional thought, for each is a coordination of 
intuitive aspect of reali! 


derived at the level of 


concrete thou; 
their concret 
concepts. 


BEHAVIOR THEORY 


vio although not 
ognitive growth in behav- 
a аге now talking of “strategies”; 

“rule”; ‘programs”’; “plans.” 
But for people who have studied behavior theory I 


would like to review a few points that will help inem 
grasp how the simplest principles of menamos re 
look to a Piagetian. First, if two schemas are ats 
quently evoked in regular succession they үг 
form a single, larger, schema comprising the ul 
and then the second in that order. The ne m ra 
larger schema is then evoked as whole. This 


i з, of course, 

“conditioning. '' The individual action is, ка 2 
с үс 

the limiting case of a schema. Sec ond, if INS ae 
more schemas are frequently evoked in irreg 


succession they, too, tend to forma larger весыш 
made up of allthe original schemas. This lar ке 
Schema will also be evoked as a whole, although x 
order of the evocation of its parts will depend ud 
the stimulus properties of the situation. Disc pt 
less, its value, as a larger schema, lies in uum 
tual inter-facilitation of its parts. This prov! b 
the basis for ‘‘trial and error” behavior thought a ; 
Thorndike to be the prototy pe of all learning. E. Sys" 
if a situation evokes a larger ''trial and error ond 
tem of actions or schema as indicated in my Ser 
point, and the larger schema includes a parton Wa 
sub-schema that is particularly appropriate to in- 
situation, the effect of accommodation will be to si 
hibit and finally eliminate the other sub-s¢ he "horn 
and so strengthen the particular sub- schema (T 
dike's Law of Effect). 


animals, which involve a linear sequence of ac oun 
But we do not know if they could be made to acc 
for understanding, for the capacity to reason an al 
understand are not functions or properties p 
schemas. From time to time a combination 9 nt” 
first two points may well give more “шеше опт 
trial and error behavior from which outsiders of acm 
lookers may infer that two separate sequences le: 
tions are equivalent to one another. For ваша 

a young child, or an older retarded child runs nof" 
the house by the shortest route whichis the gne at 
mally used. But if he finds the gate locked, 6127 
once selects the best alternative. Equivalence icit 
plied by the detour behavior, but it is not 2х0 
in the child's representation of the situation. 


«ona! 

Within these first and second order operatio 
schemas, i.e., schemas that function at the ! ^ cc. 
of concrete and formal operational thought re8P7 of 
tively - the operational structure of sy St m i 
equivalences form a sub-schema of great generó ші“ 
and enter into all processes of learning. The wt » 
sition of such schemas are essential to le arn ow” 
Such schemas do not arise merely by the pupil the 
ing if his answers are right or wrong; rather ne 
seem to depend upon an active coordination Oy d to 
subject of his existing schemas - on his abi ag" 
order these actions simultaneously and to re Cine 
nize their systematic equivalences. Good teac 
presents situations in which such coordination 
possible and necessary, and in enough variety p it 
the pupil to dissociate what is common, althous hat 
cannot ensure this. It will readily be realized noo! 
not all learning either in elementary or high SC”. 
is of this form. When a 9- or 15-year-old is ate 
sented with a problem for which he has an айе 
schema, he will assimilatethe problem (0 
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ig hy an appropriate method of attack and ar - 
lins мер This is learning of content. Не 
йе а с у а new method of coordination for 
Жас а ei had the method. Allthe problem did 
pes Strengthen the tendency for this schema to 
voked when faced with problems of this type. 


bs ‚шов. elementary ways in which schemas can 
Selina inated have just been indicated and all 
Peneta ы lend themselves to this form of 
coordina ds But since these simple methods of 
ener ea do not involve a simultaneous appr e- 
schemas Successive schemas by higher order 
tions же не connections implied by the coordina- 
actions E aoa We thus have a linear series of 
“ eannin i e first level of abstraction involving 
in the ШЕ but little understanding. Most learning 
mans TUN retarded, and much in ordinary hu- 
» 15 of this type. 


have те emphasize that I am well aware that I 

Question ацомез myself to get bogged down over the 
8rowth ic whether or not the stages of cognitive 

Child’s is I have indicated do actually exist in the 
criticized eeu growth, Piaget has been much 

Stages do over his stages, some having argued that 
Current nor exist in other aspects of growth. The 
decidin, Practice among researchers of subjectively 
Бер E responses can be fitted into Pia- 
ASSessin ema of stages is obviously not suitable for 
tive meth the validity of these stages. More objec- 
data to ds are required which would allow the 
апа to co Pte their own patterns and groupings 

rm or deny the existence of stages. 


FIND 
Nas FROM RESEARCH ON PIAGET'S IDEAS 


Th 
* Second part of my paper is about the findings 


Írom 
Piagets research that has been carried out to test 


Spac А 
the futurs time, causality, and the like. Whatever 
it can bi Olds regarding the presence of stages, 


that а ре, Said at the present that research shows 


t. 
there Ч Piaget, has been contir med, although 
Опе must bed blots on the canvas, and at times 
liga! he h © extremely critical of his work. But all 
ight on 45 done more than anyone else to throw 
Periments „g owth of thought processes. His ex- 
heoretica] ill stand the test of time, although his 
System is certain to undergo changes. 
he 
Srowth great number of studies relating to the 
bats oj Concepts shows that it is between 6 and 8 
asic eon 8€ that the pupil begins to elaborate the 


Conc 
Сап now ane of mathematics and science for he 


geri A pees the part played by himself in or- 

be me, it perience, If we take, say, the concept 

a Eins (o coon deco” amply confirmed that the child 

inl Pars o rdinate instants and intervals around 

DE time Woon although he may well have been us- 

Previously rds and telling the time for some years 
+ Likewise experiments relating to the 


overall growth of logical thought have shown the 
same general progression whether inthe area of 
science, history or religious thinking. The availa- 
ble data again confirms that it is not until 13 to 14 
in ordinary pupils and 11 to 12 inthe very able, that 
formal thought is possible and second order con- 
cepts are elaborated. I must stress, however, that 
the stages are not clear cut, and I shall return to 
this point. Indeed the stages are blurred, so that 
when a pupil is approaching the stage of concrete or 
formal thought, he is found to be ata more advanced 
stage of thinking in one situation thanhe is in an- 
other although both involve the same operational 
structure. It is also true that in some experiments 
certain stages appear to be absent; some responses 
are found that cannot be fitted into Piaget’s stages; 
and occasionally a child is found who appears to con- 
serve, say, weight before substance. On the other 
hand, there is no research that I know of, that has 
shown that Piaget was wrong concerning the se- 
quence of stages for any one situation for any one 
child. This was clearly seen in a recent study, just 
completed, where a group of boys was tested on the 
same experiments each year between lland 15 


years of age. 


It would not be inappropriate if I now gave youa 
couple of examples of protocols obtained ina recent 
study in the growth of logical thought in relation to 
history. It will keep our discussion well and truly 
about children. Itake the example from history as 
Icannot then be accused of being interested only in 
mathematics and science. Pupils read a story con- 
cerning William the Conqueror - the last man who 
successfully invaded England and that was in 1066, 
With the story still in front of him, each child was 
asked individually, ‘‘Was William the Conqueror a 
cruel man?" This was followed by ‘Why do you 


think so?” 


One pupil age 13 years 8 months gave areply 
that was clearly at the level of concrete thought. 
He said, “Well, William wasn’t cruel at the begin- 
ning because he allowed the English to collect the 
dead bodies. If he was really a cruel man he would- 
n’t even have allowed that. Yet later on, after the 
Danes had left, he’d no need to take revenge. He 
could have reconquered the North - tried to make 
treaties and things." Here the pupil could use the 
information provided, but he wasnot able to form 
a mature hypothesis from a consideration of all the 


implications in a situation. 


Another pupil age 14 years and 8 months replied 
at the level of formal operational thought. ‘It de- 
pends on what you call cruel. If the definition of 
‘cruel’ is to kill and ravishand burn for any purpose 
whatever, William was cruel. On the other hand 
if one is prepared to accept politicalneces sity , 
William’s cruelty was justified. Compared Sith 
many other feudatories, knights and so on, he was 
essentially a kind man. They ravaged generally for 
their own advantage and without care for the com- 
mon folk of the land. Duke William, if the comm 

ople went with him, seems to have been are haben 
to protect the common people from ravages. If, h е 
ever, they went against him, he seems ey Pe Ow- 
treated it as a deliberate breaking of faith and ve 
accordingly. So, by the standards of his own аста 
for we really cannot judge him by our шыта = 
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he was probably not a ‘cruel man.' " This pupil 
was clearly able to reason by implication of an ab- 
stract lead. 


One of the consistent findings in our studies at 
Leeds, and it has been supported by the work of 
Jackson (1965) — a student of Dr. Lunzer of Man- 
chester — is that only a proportion of school educa- 
ble retarded pupils reaches the level of concrete 
operational thought even at the age of 15. It has re- 
peatedly been shown to be true in respect of mathe- 
matics and science; while recently we have shown 
that the growth of English morphology in such chil- 
dren, even at 15, is less advanced than that of nor- 
mal children of 4 and 5 years of age if we accept 
Berko's (1958) data as representative of American 
children. Our findings, which will be published, 
were predicted from our knowledge of the thinking 
of this type of child in other areas. The learning 
problems of school educable retarded children, and 
of the least able left in elementary and high school, 
Stem from the fact that their schemas permit on ly 
a linear series of actions below, or just at, the lev- 
el of concrete operational thought. They learn, but 
they do not “understand.” Transfer certainly takes 
place in these pupils as it did in Harlow's apes, but 
only when the schemas involved are less complex 


than those required at the level of concrete o pera- 
tional thought. 


One great issue that research hasclearly indi- 
cated is that problems that appear to have the same 
operational structure are not all sol vable atthe 
Same time. Much research shows that both concrete 
and formal operational thought is, at first, a task 
Specific within limits, and that thinkingis not organ- 
ized to the extent that Piaget's theory would indicate, 
Dodwell (1960, 1961) showed that there was only a 
moderate correlation between tests which, in Pia- 
get's view, all involved the same level of thinking 
and which all involved integralaspects of the number 
concepts. Later Dodwell (1962) pointed out that 


and number develop with- 


Same. Even whent 


cisely the same, the quality of the c 
will vary according to the a; 
and Slater, 1960). 


remains pre- 
hild’s thinking 
Pparatus used (cf. Lovell 


It would, of course 
workers if i 


ly given data and 


Д е they cannot be ge i 
situations atonce. For exam xor e 
ceived before weight. 


ЕНИН 


and Piaget, is because it is more difficult to eed 
ize, equalize, etc., objects whose properties A 
less easy to dissociate from one's actions, Rt 
weight, than to apply concrete operations to pror 
ties that can be rendered more objective, е. А 
length. Piaget (1956) also speaks of the not 10 da 
**horizontal differentials." This suggests that i D. 
same or similar concepts when derived from dif Ба 
епі materials or situations, developin staggered se 
quence rather than simultaneously. But this notlo 
does not fit well into his general theory. 


At the level of formal thought we find m uc h ae 
same. Ina study recently completed, in which © ү. 
dren were questioned individually, we have foun D 
that the schema of proportion is not availab i s 
problems involving money, speeds, areas, d 
etc,, at the same time. Mathematical conce b ras 
seem often to be available at first in specific Sit i 
tions, even when they depend on second order en 
ational schemas. Lack of specific experience, 1 dif- 
formation, vocabulary, expectancy or individua ПШ 
ferences in intellectual functioning which are 
known, most probably all play some part. 


d 

A number of studies have been undertaken, А 
оп a neo-behaviorist approach, to see if intens can 
periods of specific training lasting afew weeks, аг 
speed up the growth of understanding of dens or 
issues; for example, the conservation of num een 
weight. The outcome of these studies has v an 
largely inconclusive, Piaget’s view (Ripple pai 
Rockcastle, 1964), which was expressed in Le ая 
is that the child may learn something of the 51 еп“ 
tion, but the training will have no effect on rs 
eral level of understanding for the specific ai ent 
is too trivial. The modification of a child's m far 
Structures in Paiget's view, necessitate а jac 
wider, more lasting, and more radical a pP! OR 
which involves many of its child's activities. 


THE WORK OF SMEDSLUND 


ill 
In concluding this section of my paper IW m 

view a few details of a recently published wiy, as 
Smedslund (1964). It was a very important Qm nal 
it is extrememly relevant to some of the poin spe” 
I have raised in this section of my paper. ne in” 
Cific purpose of the Study was to investigate |, fof 
terrelations of the Specific acquisition of abi "car? 
Concrete reasoning using different items. He e 
fully laid down a number of methodological г Td in 
Which were applied to the construction of ете 
order to maximize their diagnostic validity: tial , 
items included class inclusion, reversal of or co? 
Order, conservation of discontinuous quantitie t, е 


servation of length, transitivity of length. P tent 
nine items in all. Many of the items have SUP jare" 
All the tests were 


given individually to 160 een 
age 4 years 3 months to 11 years 4 months, © 
distributed over age and sex. 4 

i 

The upshot of his findings was that under the fio! 
€n conditions of the experiment, one cou 1d a ue 
with a fairly high degree of confidence from t ail, 
cess or failure on one sub-item to success ОГ reli? 
ure on another sub-item; that is, intra-ite m Te gg 


"m " we 
bility was high. On the other hand, the picture е 
different between items, for the inter-! 
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relations 5 

тено л меге far lower. The difficulty of the 
SE d iram 122 passes to 77 passes. When 
ес} ей, oy паг үе or failed all the items were 
озб ir à percent of the remaining subjects 
quired je tne E: r M hypothesis that the items are ac- 
thou | rder of their difficulty as measured 

tal number of subjects passed, 


Smeds 
С е. points out, in retrospect, what he con- 
Proposes HN a marked weakness in his study. He 
ing task i а пеп investigating a concrete reason- 
Percept Ms кый ig make a clear distinction between 
stimulus D S depending upon the momentary 
5 told to айа, goal object - that which the subject 
Ormed by thew and inference attern which is 
orekan jE set of premises and the conclusion. 
by the Suis 1 ine Stimulus situation as apprehended 
goal р. the percepts; quantity and length 
inference 5; and conservation and transitivity 
е Г мыш Smedslund thinks that since 
€ effect of a d influence the solution rateof items, 
т held ingle factor can only be studied with 
at other Fete The author is of the opin- 
transitivity» being equal, conservation pre- 
One goal dbi k У, but he feels that only in relation 
, Will it ed, and one narrow perceptual co n- 
“onservatio, © Possible to determine whether or not 
“Ommentat; ); transitivity, seriation, associativity, 
tion, “Uvity, addition/subtraction, multiplica- 
bos acquired in rigid predictable se- 
ere, indeed, is a field for research. 


IMPLICAT IONS 


es 


Wences 


,. In this Я 

intep is Section of my paper, I want to discuss my 

Eets ori 01 Of the educational implications of Pia- 

Search, and to suggest onr or two lines of re- 
© be helpful T have listed a number of points. 


1. Whi 
be Ero itla рхрегіепсе is always necessary for men- 
ré ern expe aget is clearthat mere submission to 
to Structure fee is not sufficient to make a child 
be be active 15 own thinking. Rather the child has 
Come away He has to act on material things and 
“Ate of the significance of his actions. 


2. qm 
* His oba 

став s €rvations led him to believe that it is 
pio Геп teng 176 in the process of organization that 
sure to repeat playfully and with seeming 
ап Ў organized ot? when such schemas have be- 
are he Schemag the apparent pleasure disappears 
me Com ined саве to be repeated unless they 
eae о some « orm new schemas or serve asa 
ER е chilg епо Learning then seems to start 
Available ' ‘Tom the schemas that he already 
le discover in ctions on his part, that is, explor- 
апа ,' АП have 5 using new ways to solve old prob- 
tuere Self-exto, intrinsic interest for the young. 
Dern 1с Motivatio nding, Naturally the level of in- 
the (20S, to the 25 Varies from child to child, due, 
central neryo Уе Of activity in certain areas of 
too f or a os System. But when the schemas 

ЧЕ Solution to some problem are not 
force Child, the is qo mplexity from those available 
Proble? o accom, €quacy of existing schemas will 
Sche * Hence modate to the conditions of the 
Mas to the child restructures his own 


eny ard 
leg Ment, савт Cognitive adaptation to his 
ing, * put he extend; Suy does the child solvethe prob- 

1 his Capacity for further learn- 


implicați п 
ation of Piaget’s findings is that we 


need to begin to stimulate the intellectual growth of 
the child in the early weeks of life. Thechildneeds 
to encounter a variety of changes of sensory experi- 
ence in respect of sight, sound, touch, and move- 
ment. He needs an opportunity to see and grasp a 
variety of objects, and later, suitable toys that in- 
volve him in a greater range of actions; opportuni- 
ties for physical movement and experimentation; 
and novelty. We must always bear in mind that 
there must not be too great a gapbetween the 
schemas available to the child and those demanded 
by the situation. Yet in spite of the help given by 
Piaget in assessing a child's level of thinking, what 
is involved in producing the correct amount of gap 
between the schemas available to thechild and those 
demanded by the situation remains vague. This is 
where the intuitive skill of the teacher is called for. 
It is his task to arrange, or find in the environment, 
problems which call forth the schemas of the child 


in new and novel ways. 


3. While language acts as a vehicle for thought 
and enables it to be carried immensely further, 
language is fitted into the thinking that originates in 
the first two years of life. If thought and language 
are to keep in step with one another as the child con- 
tinuously models his language on that of adults, the 
child must be active in building his thought proc- 
esses. If this is not the case, itis more likely that 
the child's verbalizations will lead adults to think 
that he understands more than he actually does. 
There is a grave danger of this happening between 
about 2 and 5 years of age since thechild has copied 
the adult model of language yet he remains at the 
pre-operational level of thought. One fundamantal 
lesson for parent and teacher from Piaget is that 
while language is important, mere verbalization and 
verbal knowledge are of little value in themselves. 


4. Piaget (1926) indicated that social influences 
play a role in helping a childto move from cognitive 
egocentrism to systematic thought. Mere experi- 
ence may have no effect on the available schemas, 
or an attempt may be made to distortthe experience 
so that it could be assimilated. But social interac- 
tion with the peer group forces the child to re- 
examine his own thinking, so that he can satisfy his 
need to share the thoughts of others and his need to 
communicate with them. Argument with the peer 
group forces the child to reason with himself. Like- 
wise, dialogue and discussion with parents and 
teachers, the answering of questions, etc. , contrib- 
ute to the growth of concrete operational thought. 


5. Piaget's views clearly support what is broad- 
ly called a discovery approach to learning with a 
judicious use of exposition atthe elementary school 
level. But in high school more use can be made of 
exposition since the child's ability to appreciate the 
form of an argument improves, and he is able to 
elaborate concepts not themselves derivable f r om 
first hand experience. The Geneva school argues 
that the onset of the stage of formal thought is ге]- 
ative to the culture pattern. Beyond the age of 12 
years or so, due to physiological factors, this ley- 
el of thinking may be a product of the progressive 
acceleration of individual development under the in- 
fluence of education and culture. The 13 ied 
is not only dissatisfied with the gaps and uncertain- 
ties resulting from concrete operational thou ght, 
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is beginning to think beyond the present. 
се а аке is likely to be determined by the 
experiences received at home, at sce hool, and at 
work; his social attitudes; the climate of opinion and 
expectancy in the community, and the concepts that 
are frequently made use of in the society. In com- 
mitting himself to the future, the adolescent begins 
to build theories. 


All high school pupils thus need the opportunities 
to discuss with adults and teachers, andamo ng 
themselves, viewpoints and theories relating to var- 
ied problems; the viewpoints and theorie s some- 
times being in direct conflict with one another. They 
also need the chance to see, at first hand, the kinds 
of employment that will be available to the m, and 
be in a position to think of, and discuss, their fu- 

ture roles as workers and citizens. T hey thereby 
get greater opportunities to commit themselves to 
ossihilities. The Greater the need to question and 
dina out, to Struggle ior solutions to problems, and 
to commit oneself to Possibilities, the greater 
seems the likelihood of formal thought developing. 
It may be that the culture pattern rather than the 
school plays the greater role in the growthof formal 
thought although we do not know if this is so in fact. 
Peluffo’s (1964) study, which compared childr en 
born in Sardinia but who had been in Genoa for vary- 
ing lengths of time, with children born and bred in 
Genoa, gives some indication of the likely effects of 
the culture pattern on the onset of formal thought. 


6. In the upper-elementary classes and in high 
school, extrinsic motivation begins to influence the 
pupil. For example, he may become aware of what 
his parents or society expect of him; or he may fear 
failure or punishment if he does not work hard. One 
must not belittle this kind of motivation for it often 
stimulates the child to learn and to work as hard as 
he can at the level of thinking at which he is. But 
there is no evidence that it can, in itself, change the 
quality of the pupil’s thought, Extrinsic motivation 
may well hold a pupil to a task, and in virtue of the 
prolonged child-task inter-action enable him to re- 
structure his own schemas, Thus external influ- 
ences which are at work over Я 
ав, for example, parental encouragement, can play 
an important but indirect part in intellectual growth, 


CONCLUSION 


In conclusion, I would like to suggest areas in 
Which we need research: 


1. What are the 10; 
Stimulation of the culi 
types of school educ. 


ng-term effects of very early 
turally deprived and of certain 
able retarded children? If such 
is commenced durin 


g Piaget's sensori- motor stage 
of development, how do the long-term outco mes 


utcomes when the enriched en- 
until 3 and 5yearsof age? The 
recently reported lon 


2. What are the long- 


based, as it Were, on Piagetian ideas 
When they employ materials and situa: 
sic interest to the child. 


term effects of an education 


5 particularly 
tionsof intrin- 
; from K throughgrade 4? 


000 EE 


Of course there will be some exposition, pid ood 
will be an increased рү отав арад Es ry 
ition in the u r classes o e ele 
ese pel in higi sctool. What are the ода ee 
effects of such methods over, say, 15 to 18 y id 
Is there any difference between such See ati 
those brought up on other approaches at the s 
fourth, eighth, and twelfth grades? 


í ife og- 
3. What is the effect of the emotional life on COS 


h 
nitive development? This is an important, E i 
complex, question that cannot be brushed u ‘aiid 
the 3 to 5 year period especially, fantasy - wh ing 
define as emotion clothed with images - is play at 
а role in the growth of thought. We ignore ШЙ iris 
our peril. If in real life a child can dramatiz ae 
fantasy, he may free himself from the do 
of fantasy, and thought processes m ay dev jo 
smoothly. When the 2- to 5-year-old is pres ander" 
With a Story or situation whieh be only garthy under 
stands, he brings to it his experiences and gpl 
tasies to fill in gaps and structure, say, the nance 
Each young child will have something of his OW can 
the interpretation of the story, for informan aly 
not yet be classified systematically and objec only 
In the fairy tale particularly - which again D eee 
half understood - the child's feelings of fear, (ya 
love, etc., which already exist, are pe rce " and 
by him in the story. The story acts as p ү in 
he is helped in recognizing the parts in Kapy г” 
the ‘‘Good Prince” or “Ваа Dragon. '' The “n s" at 
like properties” of fairy stories, and the nd 
he can structure it with his own experience а паг) 
tasy, perhaps explains its abiding value. SEE id Р 
the child deals with his fantasy in painting o work 
ing. At 6 to 7 years of age fantasy is still por ac 
but the child is in better control. He may Iam! 
cept a story that he could not tolerate at 4. pu i 
clinician in the accepted sense of the ter ma 0 
have enough experience with young children gr wt, 
that the fantasy life is playing a role in tho dent, 
of thought processes although we have little s | A 
in a scientific sense on this point. What d V, 
fect of kindergarten and school activities Pianta 
are likely to help the child to control his {0 ve 
What is the effect of using materials likely atici®” 
personal significance for him? The mathe. s ne 
logician may be forgiven for ignoring this t nf, 
is trying to teach kindergarten children some. jul 
about sets. The psychologist and educator mic 


realize that here is an important field about 
we know little, 


an 


d 

an 
4. What is the effect of the culture pattern gug", 
Subculture patterns on the growth of human tion р 
processes? We know something of the relati py 


; nt op 
tween socioeconomic groupings and attain’ „р 
far less about su " 


ch groupings and thought. P 4) 
from Hong Kong via Goodnow (1962), and fr 199 
Tiv tribe of Nigeria via Price-Williams (19 a eth 
suggests that culture pattern may not have nTLA 
ous effect on the growth of concrete o pera he / 
thought as on formal thought, especially if tev 
terials used are well known to the testees. e 


jc X 
theless there is some delay at the level of C07 e 
reasonin 


g due to culture pattern as the work er e 
1410 (1964) in Italy, and Almy (1966) in Aura 
What are the characteristics of the sub-cu 
that have the greatest downpulling effect? 


y 
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9. We need to know why a schema, Say. that of Jackson, S. The growth of logical thinking in nor- 
Proportion cannot be applied to a wide variety of mal and subnormal children. British Jour- 
Situations at the same time How important is fa- nal of Educational Psychology. 1965, 35, 255- 
таву With the problem, lack of specific experi- 258. 

nce in that area, and ex ations based dis | 
ous experience? Ме рвана она based 60 i Lovell, K. A follow-up study of Inhelder and Pia- 
get's The Growth of Logical Thinking. British 
. 5. We need a Very great deal of information re- Journal of Psychology, 1961, 52, 143-153. 
iw the growth ot more advanced concepts in 
""hematies a scie: эр 6 ion" 2 tan 
кору: t ei ИРЕ Lovell, K., and Slater, A. The growth of the con- 
cept of time: A comparative study. Journal 
" These Questions that I have raised are broad of Child Psychology and Psychiatry, 1960, 2; 
рез, but ones of great importance. They are of 179-190. 
ар Cance to mankind, not to just the Americans | 
е É ; f : —- 
British, теш, N. La nozione ® уау ANNE 
è КАША AL ук coc: ` 
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Promoting Creative Thinking 


in the Classroom 


THE LAST decade has seen a Sharp 


increase in the volume of researchon creativity and 
creative thinking with a major focuson isolating the 
basic processes of creative thought (Mednick, 1962; 


Mendelsohn and Griswold, 1964) and on identifying 


the unique characteristics of Creative persons (Mac- 
Kinnon, 1965; Barron, 1964; Crutchfield, 1963), De- 
Spite such unprecedented interest in Creativity, un- 
til quite recently there has been relatively little re- 
search on the general problem of nurturing and pro- 


moting creative thinking, especially in theclass- 
room setting. However, due partly to increased 


public awareness of the need to husband our intel- 
portant topic is beginning 


lectual resources, this im 


to receive the attention and research Support it de- 
serves. 


The purpose of this paper is to describe part of 
the research program at Berkeley for developing 


curriculum programs aimed at promoting the gen- 


eral level of creative 


thinking among elementary- 
School children and to 


consider 


8 Some of the pedagog- 
ical implications of such resear 


ch, especially re- 
garding the value, feasibility, and justification of 
teaching for creative functioning. For a survey of 
other current research in this area representing 
differing theoretical and meth 
Зее Myers and Torrance (196 


Parnes (1965), Suchman (196 
son (1961), 


By Way of introduction 
most of the Current atte 
ing materials, including that of the 
share two fundamental assumptions 


+ it should be noted that 


: (i) that all 


EN 


odological approaches, 
4), Osborn (1953), 
1), and Upton and Samp- 


mpts to develop such teach- 
Berkeley group, 


Martin V. Covington 


iversity of California 
University Berkeley 


А m 
children, regardless of age or initial Intel т 
level, fall far short of realizing their poten s c 
creative thought, and (2) that these individua ge 
be taught to make fuller and more Bppropr e, and 
of their capacities by means of syste mat 
direct training of certain cognitive skills. 


1 

; re? 
As for a definition of a creative child ae сбб 
sense the ultimate criterion by which the (s gent, 
of a teaching program must be judged), it i {2 ex, 
ally agreed that he is intellectually curious tin£ an 
ploratory, taking an active part in manipula ве s 
reconstructing his environment. To рагар 
number of current definitions: 


r 
The creative child prefers to explore те 
known, rather than to conserve the alre ew 
known; he prefers explaining facts in Har 
ways, rather than continuing to rely She in^ 
ditional well-established explanations; Ве е 
dulges in adventuresome thinking and ra ith 
questions, rather than being content 

things as they are. 


ng 49 
With these preliminary comments 8 е етуй ort 
stage, we turn next to more pointed pu bese 
about the nature of creative thinking and е deg" of 
to foster it — observations which to a Ion ас!е ор, 
have determined the specific form and C firs PW 
the research of the Berkeley group. AS ay he а 
we will identify and discuss several wide ѓа! 


conceptions about creative functioning ал! 
itation. 
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MISCONCEPTIONS *CERNIN 
NCE NS CONCERNING THE NATURE 
OF CREATIVE THINKING 


hg oe the most commonly held belief concern- 
ative functioning is that the child will become 
> own accord - by drawing on some 
atent potential - if only he can be placed 
ating yet permissive and nurturing en- 
The erroneousness of this view is due 
ана S misreading of the nature of man as 
ls Jaten, capi, mPlification of the means by which 
Cifically ee ACIES are to be realized. More spe- 
Sive and set eny tronment which is at once per mis- 
role in ЧЫЛЫК ШЕ undoubtedly plays an important 
Creativo Em TRE the conditions necessary for 
П Order to es ; pu this is only part of the story. 
Nosphere th e EUM advantage of an unrestrictive at- 
at constitute child must first come to understand 
W he es creative ideas in the given situation 
Sh Сап achieve such ideas for himself. In 
this cp, € must learn how to think ereatively. In 
а growing body of observational and 
гіа] evidence (Hammer, 1961; Holt, 1964; 
Schoo] child Torrance, 1965) indicates that most 
tive thinki ren are inadequately prepared for crea- 
Berke ey ng. Corroborative data gathered by the 
levels pee at the fifth-grade and s ixth-grade 
hink of 3n а conspicuous inability among students 
es, ore Y ideas, much less clever or novel 
Natur of o VOTE. there is little understanding of the 
fulness wi i біпашу and virtually no sense of plan- 
nor a aab necessary for prolonged effective 
‘Inds thie ative task, Not surprisingly then, one 
g tasks é responses of most children tochalleng- 
a rgo En іп a permissive environment, are 


ere lar I 
leatiys mach destrian and generally lacking in 


In Ord, 
skin ispose think Creatively the child must have 
Ең ls, o лы repertoire of creative thinking 
ing in existin е afew: the ability to recognize 
man evant aie Infor mation: a facility at formulat- 

sui ds Of the ee опе; and a sensitivity to the de- 

tive Pili © de 80 one can adequately judge the 

An оце is a Posed ideas. The epitomy of crea- 

tion, Tray @ ве а Sense of “‘disciplined abandonment. "' 
eq b Eve Skills such as those just men- 
pa the “‘discipline’’ component which 

Е Concert With an accepting nurtur- 

t needa Ore the child can take full advan- 
job . at hang, От and “abandon” himself to the 
restive nin ів perspective, the 
ertak, E inking is a more compli- 

phe; han ene than suspected by many. ids 
re for the rel, Providing a permissive atmos- 
Пес Шке teachi Of creative potential: itis 
от to create” the child how to make use of his 


must te г 
тов "hat kinis cach. Children to think c reatively, 


PPropriate > Piiagogical Strategies would seem 
à answe r8 are, no doubt,a number of 
Such PProach = vs to this Question. For example, 

anner th traditional one - is to teach in 

“iDlines number dn e child will come to act crea- 
Зы, Another ifferent Subject- matter dis- 

ity į o (еа trasting, yet complementary 


applied in specific subject matter areas. This lat- 
ter strategy carries with it the implication of devel- 
oping curriculum materials whose subject- matter 
is the creative thought process itself. As we shall 
see presently, it is this approach which is being ex- 
plored by the Berkeley group. 


Although there is considerable latitude w ith re- 
spect to the variety of reasonable a pproaches for 
promoting creative thinking in the classroom, there 
are limits. Some limits can be established based 
on our growing knowledge of the dynamics of the cre- 
ative process. For example, it has been well es- 
tablished that creative thinking is largely dependent 
on cognitive operations and dispositions of a non- 
logical character. Yet despite this, the notion as 
well as the practice persists of teaching for logic 
and for critical thinking as means for promoting 
creative functioning. As we shall see Shortly, cre- 
ative thinking, whether it be in science or in the hu- 
manities, involves a good deal more than rational 
and critical analysis, and to teach only for these 
functions to the exclusion of more imaginal, specu- 
lative processes is to fall short of fostering crea- 
tive thought. 


It has been further argued that teaching for sci- 
entific methods of thinking will establish a respect 
for the lawfulness and the simplicity of nature itself, 
Although the child may indeed infer from such exer- 
cises that nature is orderly and lawful, such an un- 
derstanding may curiously enough act to fetter cre- 
ative thought. More specifically, as currently taught, 
elementary school science, the humanities, and the 
Social sciences as well, do not a lways reflect the 
way scientific investigation andcreative Scholarship 
actually happen. Creative contributions to any field 
of knowledge are made by wrestling with ambiguous 
conflicting facts and data with the overall aim of 
bringing conceptual order out of chaos. However, 
as represented in many current textbooks, the 
drama of the pursuit of such order and understand- 
ing is replaced by an oversimplified, tailored, and 
predigested view of a discipline, emphasizing what 
is already known and Stressing the all-encompass- 
ing explanatory power of presently accepted theories 
or laws. Such а simple, clear-cut Presentation 
which smooths over complexity and controversy is 
designed primarily to make for more efficient learn- 
ing and retention, rather than to focus on the prob- 
lem of how the child himself can learn to manipulate 
a given set of facts, to draw his own conclusions 
or indeed to discover new facts, which is after all, 
one of the primary functions of the Creative innova- 
tor. There is little question that Such textbook pre- 
sentations will make for the most rapid, untroubled 
assimilation and mastery of the material, but it 
should not be assumed that the a Pproach which 
makes for the most efficient learning Will also be 
most effective in fostering creative i 
thinking. d and productive 


Moreover, in the way most subje 
been traditionally taught, the Piet Mar has 
lowed to practice on problems which r. 
tive modes of thought for their solut ion 
quently, he has no direct Opportunity to learn 
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maximize the chance for new insights. dn short, 
the child does not acquire the cognitive skills which 


would be most helpful in preparing him for future 
productivity. 


A specific case in point illustrating the lac k of 
emphasis on thinking creatively about facts and in- 
formation is the set of so-called ‘‘discussion ques- 
tions’’ found at the end of many textbook chapters. 
As commonly used, these questions are not partic- 
ularly helpful in promoting creative patterns of 
thought. Often the student is merely asked to think 
about these questions with little or no subsequent 
follow-up discussion of the adequacy ofhis answers. 
With minimal dialogue between the student and the 
teaching authority, the child rarely gets any syste- 
matic feedback as to what counts as merely an ade- 
quate answer as opposed to an ingenious, novel one. 
Even in those cases where the teacher finds time to 
comment on the quality of the pupil's ess ays, the 
feedback is inevitably delayed, thus reducing the ef- 
fectiveness of the exercise. Because the child typ- 

ically operates from question to question in what is 
essentially a *'cognitive vacuum” with little оррог- 
tunity to profit from his past efforts, he is unable 
to develop any internal standards of ex cellence 
against which he can evaluate Subsequent ideas. In 
effect this means he is unable to modify his efforts 
in the direction of more creative and imaginative 
Output. 


Another feature of such discussion questions- 
their placement at the end of the chapter - strength- 
ens the impression that these exercises are simply 
an appendage, added to the text as an afterthought 
rather than forming a central part of the learning 
experience itself. Moreover, their placement oc - 
curs at the psychologically inappropriate moment 
for stimulating productive ideas. By the end of the 
chapter much of the intellectual tension and drama 
build up as part of the content presentation will have 
dissipated, making it difficult to engage further the 
child’s interest. In contrast, the placement of dis- 
cussion questions should be coordinated with the pre- 
sentation of the content itself, Capitalizing on the 
student’s immediate curiosity and momenta rily 
aroused interest. 


RECOMMENDATIONS FOR A 


CURRICULUM OF 
CREATIVE THINKING 


Recognizing some of these pedagogical deficien- 
cies, what can be recommended with regard toa 
curriculum for creative thought? The first recom- 
mendation is that such a curriculum feature re- 
peated opportunities for the child to practice wrest- 
ling with complex challenging tasks of the kind that 

might, in a more complex form, engage the efforts 
of creative adults. Of course, such problems (which 
we shall call “creative-tasks-in-miniature’’) must 


be scaled down and simplified in man 
particular, reduci 


z ess to the 
would impart an atmosphere of planfulness 
process of problem solution. 


ically about 

Next, what can be said more specific ode 
the nature of these "térdutive- tasks in in of the 
their particular content апа the compe Я sroductive 
accompanying rules and strategies Ni creative 
thought? It is sobering to realize ud rs hence 
innovators of the year 2000 (only 32 years f today- 
are to be found in the elementary penes eat the 
As these individuals stand on the thresho aa by sci 
twenty-first century, they will be gantro ally ua 
entific revelations and social issues litera ld prob" 
dreamt of today as well as rge share of a = 
lems left unsolved by previous genera ti n oe ro 
meet these challenges the innovator of to ПЕ tech” 
must create new social systems, пө ни о 
niques, tools, and theoretical structur tly fash" 
accomplish this, he cannot rely on presen 1 schola 
ionable views as to what constitutes fruitfu af ord 
Ship or good scientific method, nor can е е гіі 
instill in him a blind devotion to currently aratus 
theories, since much of this theoretical арр? is 
is already approaching obsolescence. - Here аспі 
а compelling argument for the necessity 0 { епсош " 
broad strategies for creative thought and O riosil? 
aging cognitive flexibility and intellectual cu 


en! 
rather than merely teaching for what we pre$ 
take to be true or useful. 


п 
ce? 
What kinds of tasks could children prac Б 
now to prepare them for their roles as ee ne 
tors of the future? Two kinds are Suggest Th 
hypothetical problem and the enduring pro t Тасі 
former type minimizes reliance on prose the latte, 
and theories in obtaining a solution, while n 
type focuses on the deathless issues of ne eres e 
fairs which are certain to be the subject of ture аб 
reinterpretation and re-evaluation іп апу {Ч 


pie? 

Typically in the case of the h thetical РЕ ү 
certain artificial conditions are postulated йму 
not correspond to present realities, and БЕЯ nal! 

ual is called on to operate within these К, " 
consistent but artificial systems. One pro pe 


{Л 
asa” 
this type, which itself has become popular 4 iven 
ure of creativity, is the consequence test: gam 1 
certain conditions, such сёй 


as a world in which y 
runs backwards, what are all the consed? абу 
Such problems demand cognitive flexibility», re 
nation, and the ability to adopt as temporar 

а set of foreign and unfamiliar assumptions: 


or 
At Berkeley we are now drawing up plans f 
experimental teaching apparatus which sim" Ue opt 
visit to a strange planet. The student is his i 
charge of the first exploratory expedition. e 
a highly creative and demanding опе, is (0 tra 
understand and to explain the welter обв U 1 
events which he will encounter in this а11ё 0ай 


1 ] 
miliar world where the usual physical, biolo rel; 
and social laws are suspended or greatly ™ ny Pig’ 
Here the child is literally confronted with а ^ij "n 
thetical world which, although it is ordered 4” pwy 
ternally cons 


istent, operates in ways th? , ИГ 


counter to his previous experience. He m d yo 
pend his present expectancies about the жоно | 
knows it long enough to create new explana by jj 
Strange and perplexing phenomena. Thus usw? 
mitting the child to deal with hypothetical tas 


S 
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Б us Creation of new systems of ideas, inde- 
ede of present facts and assumptions, he can 

thee. mest the time when these facts will outlive 
o ulness and will no longer serve as a basis 
T productive thought. 


eventa ME the largely unknown course of future 
Problej * are nevertheless certain enduring 

nS of man and Society whichare constantly 
Selves and no doubt willengage the 


re creative thi V = 
dent of олау осор уе thinkers. The young stu 


he Search for new implications and 
m in contemporary terms. Many 
е most Suitable teaching examples are found in 
ing with D Such as the Greek myth of Sisyphus, deal- 
tragedy of T. S reluctance to accept his fate; the 
of man’s rankenstein, reflecting the consequences 
being is hee Old dream of creating a more perfect 
Sonifying ta Own image; or the mythical Icarus, per- 
regardless an’s unflagging zeal to explore nature 
understooq th © consequences. Of course, it is 
ing examp) ^t Such issues cast in the form of teach- 
and m, aS eS would have to be greatly simplified 
Standing “propriate to a child's level of under - 


ар Otice, however, th ificati 
ade , r, that such modifications 

hieleg г r 481ег by the fact that the traditional ve - 

the © dramatization of these issues are 


play. nq S Peasonably constant - the novel, the 
id ы г ай ~ providing already familiar 
Sas, рт ui Which children can ex press their 
M a sw tunnection, we shall presently de- 
1e ent c eates ‘ve-task-in-miniature’’ in which the 
tue d recy а play dealing with yet another time- 
ly ®cuseq Tring theme — the plight of the wrong- 


1 
ап Сота аге the sources on which one can 
St ategies ating a set of broad cognitive skills 
сер ог PU Could be taught now to prepare 
Blea Such geni. Productivity? Information con- 
ing neq ont neralized cognitive strategies canbe 
frg, th Comp] У from laboratory research deal- 
u s puma Solving and thinking but 
th indi €-history accounts of highly 
Ае is a stea saals 35 well Inthefirst instance 
th Psycho}, y accumulating body of knowledge 
ih of factors Bical laboratory concerning the 
inf] 1, oth facilitative and inhibitory, 
e le urse and quality of roductive 
lem Qus effect aaely documented еха ае the del- 
Чуд Olving (Lucy Eid mental set оп effective prob- 
Ч 5 kid 1998). Here, because Ње in- 
Seay and i; wiates or perceives a task in 
Prope from argent biasing way, he is effectively 
Ое ate thinking aining a solution. Anumber of ap- 
тару Search fing, ategies incorporating these and 
ta] With refer ings can be formulated. For ex- 
em pl Set, the BET to the present example of 
апа кафу, tor Udent can be taught to view a prob- 
TeSist ju ua ase questions in new ways, 
The 8 to premature conclusions. 


Other 
a Sour, " 
nec сетор Cognitive Strategies comes 


rk de; 
of insig (Ghiselin лены ОПЗ of creative individuals 


2; Koestler 1964) 
Я . А number 
out amenta] nature of creative 

Such observations as well 


as the discovery of important phenomena associated 
with creativity, such as incubationand intuition, As 
an example, the famous and widely cited self- re- 
ports of Poincare, the French mathematician, have 
served to focus attentionon the conceptually fertile 
notion of creativity as basically a process of recom- 
bining ideas in new and novel patterns. And of most 
relevance to our immediate concern, such observa- 
tions have led to the elucidation of a number of spe- 
cific problem- solving strategies, among them the 
technique of identifying the essential elements ofa 
problem situation and deliberately juxtaposing them 
in various ways (Campbell, 1960). 


By combining the experimental laboratory ap- 
proach and its virtues of empirical validation w ith 
the rich and broadly inclusive self-reports of the 
highly creative thinker, one can formulatea set of 
rules and strategies which are pervasive enough to 
apply to a variety of creative tasks, yet simple 
enough to be readily understood by ele mentary- 
school children. 


In summary of the first section of this paper it 
has been argued that before the student can take full 
advantage of a permissive stimulating atmosphere 
he must be taught how to think creatively. In devel- 
oping a teaching program for creative thought the 
single most important pedagogical issue is the long- 
term social and personal usefulne ss of what is 
taught. It has been suggested that one reasonable 
teaching strategy, in light of this consideration, is 
to strengthen those cognitive skills basic to all cre- 
ative thought in the context of complex yet meaning- 
ful problems which reflect the principal steps in the 
creative act. These observations, taken as a whole, 
have formed the broad guidelines for the research 
carried out by the Berkeley groupover the past 
several years. 


THE PRESENT RESEARCH 


As a first Step it was decided to develop a set of 
prototype instructional materials dealing with only 
one aspect of productive thought- creative problem- 
solving. If the particular teaching approach showed 
promise, then other programs would be designed 
focusing on yet other domains of creative thou ght 
such as creative innovation and creative ex i 

— SXpression, 


After several years of intensive effort involvin 
numerous pilot tryouts, successive revisions od" 
several full- scale experimental Studies, the Gen- 
eral Problem Solving Program was developed (Gov 
ington, Crutchfield, and Davies, 1966). The Ge v- 
eral Problem Solving Program (Gpgp) consists of 
aseries of sixteen “creative-tasks-in- i а 
ture” designed for the fifth-grade and sixth- minia- 
levels. These problem episodes actas 4 а ры 
which the student practices а number of Broad b 
and strategies concerned with various f. 
fective problem Solving, such as stati 
lem in an open and unbiased fashion 
the problem in terms of familiar me 
analogies, or developing a Systemati 
erating ideas. 


rules 
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to explain, such as the puzzling behavior of a group 
of aquanauts during a deep-sea dive. This general 
theme - the mysterious occurrence- was chosen 
because it encompasses in a naturaland uncontrived 
manner many of the fundamental aspects of the prob- 
lem-solving process. In addition, such а theme is 
neutral with respect to orthodox curriculum mate- 
rials. This makes for a broader transfer of what 
is taught than might otherwise be the case if the 
principles and concepts had been associated prima- 
rily with a specific content area. Finally, children 
exhibit a keen interest in working on problem situa- 
tions featuring a mystery-detection theme. 


Each lesson is presented in booklet form and is 
cast in a self-instructional linear format. (For a 
theoretical discussion of the potential uses of pro- 
grammed instruction for fostering higher- order 
thought processes, see Crutchfield and Covington, 
1965.) Briefly, the child works through the lesson 
in a step-by-step sequence by himself and at his 
own pace. As the problem unfolds page - by- page, 
with the advent of new facts and clues, the student 
is called on to reformulate the problem in his own 
words, to list questions he considers crucial to the 
solution or to devise a plan of action which he in- 
tends to follow. The student receives immediate 
guidance in his efforts on successive page s of the 
booklet. This guidance typically takes the form of 
presenting a range of ideas, questions or courses 
of action which the student might have thought of in 
the given situation. Here the emphasis is on en- 
couraging appropriate diversity in the student's 
thoughts by showing him within wide limits what 
constitutes valuable ideas, crucial questions and 
fruitful ways to formulate problems. 


The GPSP is also designed to promote beneficial 
attitudes toward productive thinking. One of the 
basic strategies is to increase the child's ex per i- 
ence in coping successfully with thought problems. 
Each problem episode is Sequenced so that as the 
student works through the booklet he 
gressively to more clues and hints. 
child, independent of his initial capacities or per- 
sonal reservations about thinking, willat some point 
come to discover the solution for himself. In addi- 
tion there is a Systematic attempt to foster by 
means of appropriate teaching examples, a number 
of attitudes which favor effective problem solving 
such as open- mindedness, persistence, and suspen- 
Sion of premature judgment. 


is exposed pro- 
Thus,each 


A story-line is maintained thro 
ughout the GPSP 
to supply a sense of continui 


UE to provide the reader witha set of iden- 
Cation models. The stor 

eileen, er ats y concerns two school 
extra- 


es by their uncle. 
g a high-school science 
as a detective. 

The identification 

е - model techni is i 

introduce the reader gradually ie he ра 0 


3 o the difficult and 
tive dbi. ee Process of becomin 


ig а more effec- 
. For example, the roles of Jim and 


Lila reflect the vicissitudes associated withallcom- 
plex problem solving - the long discouraging р Єт 5 
ods of intense effort with little apparent progress. 
the inevitable setbacks, and finally, the intellectu 
elation of discovering a solution. Through Jim к» 
Lila the reader can experience all this vicarious | 
without being plunged immediately into the B 
After the first few lessons, however, the PEROT n 
gradually drawn into the various problem-solving n 
tivities by being requested to think of and to rec d 
his own ideas in concert with the efforts of qum n 
Lila - first the student generating his evade s. 
questions, then Jim and Lila responding with 1 ara 
The feedback examples provided for the reader à 
presented as Jim's and Lila's ideas. 


Against this backdrop of transitory day ed 
successes and failures, a long-range change over- 
place in Jim and Lila. They are depicted де a 
coming initial handicaps of reticence, арау р not 
negativism toward thinking until ultimately. ББП“ 
without an occasional setback, they become uer 
ably comfortable with their own thought proce 1 sit" 
much more enthusiastic about tackling problem. ip 
uations, and more confident in their own abili er^ 
cope with them. It is hoped that this subtle DU. 
ceptible change in Jim and Lila will foste! кетеп" 
reader a sense of hisown progressive improve’ gers 
At the same time it is intended that as тапу ren 
as possible will finally come to surpass J 1 mater W 
Lila in problem-solving proficiency and willo ide? 
any earlier dependence on them as a source O 
and inspiration. 


the 


ila 
The inclusion of the uncle provides Jim anne! i 
and presumably the reader as well, with а iy ur 
lent authority figure and confidant who not pn nes, 
tures and encourages the nascent attempts e ink 
tyro-thinkers but who also shows them ПОМ. ард, 
for themselves by means of appropriate ru = а vil 
Strategies. Additionally, the uncle stands s telle? 
ued model personifying an enthusiasm for 
tual exploration. jl 
or, 
Each lesson is presented in an illustrated is m, 
Such a dominantly visual presentation not on e p" 
creases the student's interest in the materia gil 
makes it easier for him to follow the nese D» 
complicated dialogue and thought sequence? б praf, 
Story characters. The lessons contain an 4 at v 
of 40 pages and are designed to be presente MU 
rate of one lesson per day. The average nd e? 
to complete a lesson is approximately 39 ™ 


T cor 
Several separate studies (summarized ЇЇ gut "ni 
ton and Crutchfield, 1965) have been carrieC p^ 


ing various preliminary editions of the ern af 
to date have involved a total of some 481 eat Du 
and sixth-grade school children from the тө ut gy, 
cisco Bay Area, of whom 267 were give? ш use? г 
ing materials and the remaining 214 we? €. es ie 
controls. The basic design of these studi aij 
sentially the same. All students are first Sus с 
tered an extensive pretest battery (si* папи 
length in the latest study) which consists 0 


of tests of creative thinking (Torrance, 66c 
of problem-solving ability (Covington, 19 al 
ventories designed to measure the child 9 

toward problem solving and thinking colton» e 
1967), and various tests of school achiev? 
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Ope pe ability. These data are used as a basis 
ual diae matching of the cla ooms. Individ- 
room dion Es are matched in pairs; one class- 
ike Быйан, Pair is then assigned at rando m to 
Condition : Condition and the other to a control 
a fisting a Bach of the various studies to d ate. 
the GPsp bs EKroups have always been administered 
trol conditi а вов аву, а number of different con- 
Potheses . OnS have been used to test varicus hy- 
А 5 about the nature ot the training effect. The 
še, however, and the one on which the 
sed, calls tor the administra- 
ar but shorter set of self-instruction- 
olving те ngae content is unrelated to problem 
^ Purpose of the control program is to 

the Pens in er involvement and participation on 
Css. uS control children and to make certain 

Y are wel] acquainted with the self-instruc- 

с ЕНШЕ Following the training period all 
includes ща ae ministered à posttest battery which 
ests of propr n of the attitude inventories and the 
50 em- solving ability along with parallel 

е tests of creative thinking. 


Severa] 9 


Benera] amples will suffice to illustrate the 
Sttest ypes of tests included in the pretest and 
Í si Patteries — their content, style and degree 


Песка parity to the training material. One type re- 
In these į morg traditional tests of problem solving. 
Strie iong 2141Се5, because of a large number of re- 
Set o ans Placed on the task, there is only a limited 
ist е Swers 9r principal solutions which will sat- 
is pla ed тоет in its entirety. Here an emphasis 
М ће St dent | 1 Оп Convergent and divergent thinking. 
Е Propriate ia Called on to diverge in his search for 
the 7 Put at the. ~ Opening upa host of possibili- 

Ч fe mo e Same time to converge, focusingon 
Ple ig the St suitable ideas. A representative sam- 


ao Prop 08у problem (a modification of the clas- 


“quire (mused by Karl Duncker). The child 


inei dos 
tre! a jo invent à method to kill a tumor deep 
Sty Surround БУ Using an X-ray but without harming 


Gnd. 
Боан is tha E healthy tissue; The primary con- 
oth 1 the ealth the X-ray is too strong it will kill 
Sr hand hs tissue and the tumor. If, on the 
тор the good t; -ray is made too weak, it will not 
Sessi Incidenta e? but neither will it kill the tu- 
wite "8 а high аеру is problem in addition to pos- 
Unlike an thine of curriculum relevance is 
8 in len ing encountered in the training 
Th NS of content or theme. 
ris S Other ma; 
tive tion atten type of problem included in the 
quiz aking (T 55 is the Minnesota Tests of Crea- 
га Open-ende i.rance, 1965). Such problems are 
be jug Mber of Vith few constraints. This allows 
ing geq as the Solution- ideas, no one of which can 
duct- im 108 Suitable. One example featur- 
toy dop Of all Provement theme requires the child 
9g Ingenious clever ways to make a 
T. ? to play with, 
Vario, ^O the reg m 
ons ies ph ҮП eneral, the outeomes of the 
Children Ent. Phe Ving the GPSP have bocs high- 
dren 15 markeqiP ^ Íormance of the instructed 
апа on РОЊ on he, Superior to that of the control 
the tests of = et of problem-solving ability 
a$ ative thinking. (For a detailed 
ell as a discussion of scoring 
On, et. eL, 1967, ) 


In order to gain a clearer picture of the over-all 
magnitude of this training effect, consider several 
performance indices such as total ideas generated, 
total quality of ideas, mean quality per idea and the 
incidence of principal solutions. Composite scores 
for each of these indices are computed for every 
student by summing over his performance on a num- 
ber of problem-solving tests in the posttest battery. 
In the same manner a similar set of composite 
Scores is obtained for the tests of creative think- 
ing. On comparing the mean values for the instruc- 
ted and control groups on any of these composite in- 
dices it is found that the instructed groups are in- 
variably superior. In most cases the magnitude of 
these differences is absolutely large and beyond mere 
statistical significance. This is illustrated bythe 
fact that when comparing frequency distributions for 
each composite index, the fiftieth percentile of the 
instructed groups typically falls around the seventy- 
fifth percentile for the control groups. 


Another way to gauge the degree of educational 
significance of a training effect is to co m pare the 
level of performance of instructed and control chil- 
dren of differing IQ levels. In this connection it has 
been found (Covington, 1965; Covington, 1966b) that 
the mean performance of instructed children with 
IQ's below 99 (mean IQ = 91) isona par with the 
mean performance of control children whose IQ's 
fall between 100 and 115 (mean IQ - 107). This in- 
dicates that the administration of the GPSP makes 
for a substantial boost in the performance of low IQ 
children over a wide variety of test problems, many 
of which are curriculum relevant. 


To determine how long these training effects per- 
sist, a follow-up test battery was given five months 
after the administration of the posttest in two differ- 
ent studies, These follow-ups included a pproxi- 
mately 80 percent of the fifth-grade children from 
the original samples who by this time were in sixth- 
grade classrooms with new teachers, Wherey er 
possible every precaution was taken to insure that 
the students did not recognize the tests as a part of 
the earlier work. For example, the childre n re- 
corded their ideas on regular school Paper, rather 
than in a standard booklet of the kind used in the 
original study. Allthe tests were administered by 
the teacher herself, and whenever Possible were jn- 
troduced as part of the regular classroom work. 


In the case of tests of problem- solving abilit 
the instructed children continued to surpass their 
control counterparts on the various performance jn- 
dices, and although the margin of differences had di- 
minished somewhat as compared to the magnitude " 
effects found at the time of the original posttest the 
differences were nevertheless found to be statisti- 
cally significant. The picture isnot as cons à 
for the tests of creative thinking. Ing ne sa 
witha total of 108 children, the instr 
superior to the control grou ‚ wher i 
slightly larger sample these differences nano ped 
It appears that after a five- month interva] th a oe 
ing effects are at best marginal in acia 
tive thinking tests. This is not su 
is realized that аза group these t 
type of problem not directly train 
Consequently they would be more 


istent 
mple 
ucted group was 


е Sts represent à 
edtor Y the Gpgp 
likely to reflec 
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greater diminution in training effect than would the 
tests of problem-solving ability which are more di- 


rectly amenable to the kinds of Strategies taught for 
originally. 


ISSUES OF VALUE AND 
JUSTIFICATION 


One of the most fundamental questions raised by 
the present research is whether programs designed 
to promote creative thinking are actually needed, 
seen in the perspective of a child's total educational 
career. No matter how beneficial a given teaching 

untutored children 
er or later catch 
process of intellec- 
Б experience, Inthe 
Оп in the magnitude 

hs after the adminis- 
this contention. Of 
more permanent - and 
1 - changes will occur 


na performance superior- 
ease that margin as the pro- 
gram proceeds. While it is true that the trained 
and the untrained studen 
pansion of intellectual c 
doubt that the untutored 


use of their burgeoning capabiliti 


later age 
any greater degree, 


ities to 
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which predispose the student to undertake cre; 
tasks in the first place, rather than to st 
creasing his sheer competence for perfor 
means of ‘‘skill training. ” 


ative 
art by in- 
mance by 


FUTURE RESEARCH PLANS 


Research plans for the immediate future take two 
main directions. The first of these involves further 
development and expansion of the GPSP. In one in- 
Stance a set of supplementary exercises is being de- 
veloped to provide additional practiceonthe various 
Skills and strategies taught in the lessons. These 
exercises are primarily for remedial purposes and 
can be used when needed, depending on the r ate of 
progress of the individual learner. Another project 
concerns the development of a set of **curriculum- 
link-units,’’ that is, lessons designed to illustrate 
how the student can apply the skills and strategies 
taught for in the GPSP to his actual school work in 
Science and the social studies. 


The other main direction of research involves 
the development of new self-instructional teaching 
units which introduce the student to domains of cre- 
ative thinking other than problem solving. One ob- 
viously important area is that of creative understand- 
ing. Here, typically, one is confronted witha series 
of complex events or occurrences which in the in i- 
tial phases of investi 


lated. The task is o 
lationships between such events and in some 


Such 
forclosure 
btained. On the other hand, cre- 

not only encompasses the dis- 
covery of workable ideas, but more often involves 
an ongoing procedure of checking and re-checking 
the validity of the proposed solutions against an ac- 
cumulating set of facts and information. 


individual invents or creates new problems, new 
Systems of thought, or new products which did not 
exist р; 


reviously. One example of a prototy pe training 
eing developed involves the creation 


The boy, who is 
lassmates of this 


vr 


PROMOTING CREATIVE THINKING IN THE CLASSROOM 29 


one's emotions besides direct aggression and that 
intense feelings can serve as a basis for the devel- 
Opment of useful and satisfying products. The par- 
ticular theme -the plight of thewrongfully accused - 
was chosen because it sounds a note of high drama 
and arouses immediate interest and sympathy in 
children who typically havea keenly developed sense 
of fair play. Moreover it represents - even when 
placed in such a simplified context — one of the en- 
during issues of man and Society which de ma nds 
new solutions from each succeeding generation. 


The same general pedagogical devices are used 
here as in the GPSP. The student is led to develop 
the play in a step-by-step sequence, while thinking 
of his own ideas and being provided in turn with ap- 
propriate feedback and redirection of his efforts. 
In the early lessons the child works throu gh se- 
quences dealing with the creation of the basic plot. 
The student is encouraged with the assistance of 
identification models in the Story to generate 
ideas which seize on the malicious incident asa 
point of departure from which to explore the more 
general implications of being wrongfully accused. 
This is in contrast to the more pedestrian and com- 
mon- place approaches of si mply dramatizing the 
boy's innocence or holding the farmer upto ridicule. 
Basic to the success of this teaching sequence is the 
introduction of metaphorical and analogical modes 
of thinking. Once such concepts are grasped by the 
Student he is able to go far beyond a Strictly literal 
interpretation of a Situation and can generate a vir- 
tually limitless set of implications, 


After the outlines of a clever insightful plot are 
Secured, the student explores a number of detailed 
matters, all of which are important tothe final prod- 
uct. In one lesson he creates ideas for the most ef- 
fective Scenery and staging, given a limited amount 
of material to work with; in another he is called on 
to create segments of the dialogue. The last lesson 
in the series simulates the public reaction to the 
play which represents a final confirmation of all the 
preceding activities. Various members of the audi- 
ence including the farmer and the parents of the ac- 
cused boy react to the play. There is general rec- 
ognition that it was worthwhile in its own r ight as 
well as being helpful in c learing upa community 
misunderstanding, 


FINAL PARADOXES 
AND DILEMMAS 


А Inherent in any attempttoteachtor creative func- 
tioning are a number of dilemmas and paradoxes 


which give rise to both methodological and theoreti- 
cal difficulties. 


recurring dilem: 
representative 
rather because 
ordinary challe 
ing materials f 
а asic to the act of teaching is а guidance 


i or is necessarily guided, shaped, and 
red g , ped, 
fever eo a teaching authority. The student 
certain 1 complete freedom. He is exposed only to 


2n learnin 
termined Sei s 


these expe quences, He is encouraged to talk about 


creative functioning, then the child will inevitably 
be subject to some form of guidance. To the extent 
that an educational theory stresses the *'discipline 
component” of creative thought, any forthcoming 
curricula will be more or less restrictive and pre- 
determined. How then can we reconcile these pro- 
crustean-like features of teaching with the fact that 
in the last analysis creativity is primarily charac- 
terized by imaginal freedom and a Spontaneous indi- 
vidualization of thought? Perhaps the answer, most 
simply put, is to teach for Structured spontaneity. 
One technique adopted for this pur pose and em- 
ployed in our current work is to give the student di- 
rect practice in discriminating among ideas which 
are both unique and appropriate to a given task and 
ideas which are merely bizzare. In sucha fashion 
the student learns to limit the reachesof his sponta- 
neity. Another technique which requires a relative- 
ly long term training program involves comprehen- 
sive guidance in the early phases of learning. But, 
as the child becomes more proficient in the various 
Skills and strategies, the rigid guidance of the pro- 
gram is gradually reduced. In this manner the stu- 
dent comes ultimately to rely onhis own resources 
and initiative with only occasional redirection from 
the program. As the Student becomes more selt- 
sufficient he can experience yet another dimension 
of intellectual independence - that of determiningfor 
himself which of several creative tasks he will work 
on from among a number of alternatives. For ex- 
ample, in the case of the creative drama unit just 
described, the child could be given the option of 
either writing dialogue or of developing the scenery 


and staging, once he had completed the introductory 
units. 


are antithetical, perhaps eve: 
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Covers research conducted through 1965. Sum- 
maries of our more recent research activities 
can be found in Olton, et. al. ‚ (1967) and Olton 
and Crutchfield, in press. 
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— Measuring Motivation in 
à Culturally Disadvantaged 


School Children 


A COLLEAGUE of mine rece 
Spent a semester in seventh gr 


Írey, 1965). In a School servi 
vantaged neighborhood, he tried t 


Server. His account of this expe 


ple incident: 


Harry had been over at the confectionery 
across the Street, and had come out with the 
Test of the boys on Tuesday, when a fight be- 
Бап on the Playground. There had been con- 
Siderable excitement and Mr, Inman (the 


Principal) was trying to calm everyone down. 
Harry had left hi 


J ery and wanted 
to get it, 


This 
doesn» commonplace i 
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ү е utter 
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rience is very dis- 
couraging to a person who is primarily interested 
in human motivation. Take, for instance, one sim- 


ncident in itself is minor and 
Ct of the whole report where 
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their futile attempts toas- 


appropriate Ways, they cre- 
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ate an environment where even the teachers and ad- 
ministrators are forced to act in Ways that are not 
of their choosing. The result is a constant struggle 


between pupils and the school staff to obtain the up- 
per hand. Teachers search for 


powers beyond their control, 


Two more or less disconnected things Stand out 
for me in approaching the problem of motivation in 
School children, First i 


The second thing that Strikes 
the loose strands that may have 
eling both practical and theoreti 
knot is the notion of powerle 


me is that One of 
potential for unray- 


Cal aspects of the 
55пеѕѕ, which i 


ivati S, after 
all, a motivational concept. 
What I have to report then, is no breakthrough 
but rather a modest beginning that attempts to di Je 


cover whether certain aspects of motivation can be 
measured in a population of culturally disadvantaged 
children, and whether some of the 


motivation 
cepts akin to powerlessness e 


al con- 
an be useq 


32 


Actually we will discuss three different but re- 
lated things. First, I will present some notion 5 
about powerlessness and what I will call the “Огі- 
gin-Pawn”’ variable and discuss very briefly some 
relevant laboratory studies that we have completed. 
Second, I will shift to a discussion of achievement 
motivation and some of the conceptual and scaling 
difficulties that it presents. This is necessary in 
order to introduce a third section dealing with data 
on the achievement motive in cultur ally disad- 
vantaged children. These three sections may seem 
a bit disjointed. What holds them together is the 
relationship between the concepts to be presented in 
section one and the achievement motive . At the 
present stage of the research there is little to re- 
Port on these concepts except laboratory studie 5, 
but we do have evidence оп achievement motivation 
from culturally disadvantaged children. Asa result, 
Ihave more or less arbitrarily put the two together 
With a discussion of some theoretical problems with 
the measurement of achievement motivation. 


POWERLESSNESS AND THE ORIGIN- PAWN 
VARIABLE 


In the anecdote related above, Harry was frus- 
trated by his powerlessness to influence an impor- 
tant aspect of his life vis-à-vis a school official. 
He was a pawn to forces beyond his control. In 
much of his experience in life, and espec ially in 
School, he is, in fact, and is expectedto be, a pawn. 
Learning seems to demand that he react to outside 
influences in the School, and one way to insure that 
he will react in the proper manner is to reduce his 
freedom to a minimum, to force him to study, to 
press him into the mold expected by the school. 
Techniques designed to do this demand thatthe teach- 
er and the school have most of the power and that 
power be taken away from the student. 


Motivationally 
Picture. The s 


Specific power si 
ferred to non- 
broader goal? 


sumption ig ce pur everyday interaction, The as- 


has some autonomy 
own be 
one word Ша havior. 
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lessness, but origin behavior implies freedomof 
choice, not necessarily the ability to wield power 
over others. 


Having sketched the ‘‘Origin-Pawn’’ dimension, 
let us note that it is essentially a phenomenological 
given in the perception of behavior. As such, it may 
be a myth or a mental way station, to use Skinner’s 
railroading metaphor. To extend the metaphor, we 
might say that the main line of behaviorism runs no- 
where near this way station, and efficient operation 
of the line would suggest that the station should be 
dropped. In fact, scientific psychology has been at 
pains to discredit the assumption that a human being 
can be the ultimate origin of some of his behavior. 


Behaviorism, suffering under the strong influence 
of operationism and physicalism, has done its best 
to reduce the concept of motivation in psychology to 
something of the order of ‘‘the study of the causes 
of behavior. ’’ The trouble with this a pproach is 
that trying to elucidate the concept of motivation by 
means of the concept of cause is like trying tounder- 
stand the behavior of a human being using concepts 
found to be inadequate to explain the behavior of bil- 
liard balls. The precept that all scientific state- 
ments must be based ultimately on statements about 
physical operations has great merit, but the hidden 
implication that explanations of physical motion (in- 
cluding behavior) are reducible to the laws of me- 


chanics has seriously retarded the development of 
theories of human motivation. 


Stripped of its hidden assumptions, the neo-be- 
havioristic approach to motivation as the cause of 
behavior is unacceptable to anyone and has never 
been seriously held by anyone. It assumes that the 
human being is like a billiard ball and his behavior, 
like that of the ball, is entirely the result of exter- 
nal sources forcing him to behave. Another object 
hits him and steers him in a certain direction ~ an 
explanation that would account for certain trivialbe- 
haviors on the football field perhaps - or an exter- 
nal noxious stimulus repels him like one negative 
charge repels another, or an internal condition 
(e.g. , drive-stimulus) produced by externalcircum- 


Stances (e.g., food deprivation) drives him ina cer- 
tain direction. 


Confronted with the phenomena of motivated be- 
havior which in this framework appears akin to a 
sudden inexplicable movement of abilliard ball rest- 
ing alone on a perfectly level table, psychology has 
ingeniously devised many schemes to explain this 
untoward behavior in terms of so me mechanisms 
involving a causal chain of events and ultimately re- 
ducing to some undetected source of physical energy 
closely connected with, or actually residing in, an 
impelling stimulus (e.g., the cue bal). The cue 
ball, or impelling Stimulus, has been variously 
described as a need or drive (Hull, 1943), a drive- 
Stimulus (later Hull and Miller), incremental stim- 


ulation (Mowrer, 1960), or an affective Stimulus 
(Young, 1961; McClelland, 1951). 


In the face of all this theor iz, 
Gilbert Ryle (1949) 


based “science of behavior, ”” 


- namely to shun the 
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notions of causality and motivation altogether. Al- 
though I suspect that he may have led some to hide 
their motivational and causal notions under the con- 
cept of reinforcement, his strict adherence to the 
definition of a reinforcer as so methin g that in- 
creases the probability of a response does, in fact, 
rid the science of behavior of all of these irritating 
problems. 


By now, I hope to have laid the groundwork so 
that two conceptions of man as a motivated being 
are clear. The first is the one we use inour every- 
day dealings with human beings: namely that the 
cause or reason for a person’s behavior is sought 
first within him. More often than not, he is the or- 
igin of his behavior and is to be held responsible for 
it. The second is the explanation from scientific 
psychology that seeks a physical source external to 
the person, or at least a stimulus within him, that 
drives him. This latter notion has never been sat- 
isfactory in accounting for the phenomena of human 
motivation and has, through its emphasis on demon- 
strating a physical basis for all behavior, led to a 
psychology of learning and motiv ation that con- 
ceives of the animal and the human being as a pawn 
to be manipulated by reinforcement schedules, 


Accepting at face value the psychological fact that 
people perceive themselves and others as origins in 
most instances is, for me, the starting point of a 
truly psychological approach to motivation, and de- 
mands a level of analysis quite different from the 
causal chain analysis that attempts to reduce the ex- 
planation to physical events, I have tried, there- 
fore, to build on the assumption that the origin-pawn 


dimension makes a difference that is measurable in 
the behavior of a subject. 


The огіріп- pawn dimension has its roots inthe 
concept of locus of causality used by Heider (1958). 
His emphasis is on the perception of causality, and 
the result is that the concept is applied in "ep 
of person perception. In perceiving another per- 
Son's behavior, we do attempt to assess whether the 
locus of causality is internal or external to the per- 
Son, and this distinction affects our perception of 
the person, the characteristics that we attribute to 
him, and the way we behave toward him. 


Several 
Strated the 


Causality (Thibaut anq Riecken, 1955; Pepitone, 
1958). In our own research, we have demonstrated 
(deCharms, Carpenter, and Kuperman, 1965) that 
origin- pawn variable in 
their perceptions of others. We found that the per- 
ception of a person as an origin or as a pawn isa 
function of Several variables, such as whether some 
agent is attempting to influence the person's behav- 
tort not. Origin or Pawn perceptions are a func- 
аё Aa Characteristics of the influence agent, such 
азу ег itisa large group, a small group, or an 
Pergon ^b and whether it is liked or disliked by the 
that а у Мові interesting, however, was the finding 
TSOnality characteristic ofthe subject 
along the tected his perception of another person 
Origin- n x 
еазиге, оп Pawn dimension, Subjects were 
and Liver 5 асе developed by Rotter, Seeman, 
Es 962) that attempts to measure the 
E of Powerlessness or efficacy of the 


person. Since Rotter uses reinforcement 
terminology, he calls the Scale a measure of control 
of reinforcements and names it the I-E scale stand- 
ing for internal or external control. There isa sub- 
tle difference between this concept of control of re- 
inforcements and our O-P dimension, but for now 
the similarities may be noted. Our data showed 
that subjects who reported on the Rotter I-E scale 
that they felt in control of their Sources of reinforce- 
ment perceived other people more as Origins, 
whereas subjects at the other end of the I-E scale 
perceived other people more as Pawns. 


Since our primary interest is motivation, we next 
asked the question: ‘What effect does feeling like 
an Origin, as opposed to feeling like a Pawn, have 
on the person's own behavior ?' With this question 
we moved from the perception of others to the per- 
ception of self as an Origin or a pawn. Two labora- 
tory studies (Kuperman, 1966; deCharms, Dougher- 
ty, and Wurtz, 1965) were devised that attempted 
experimentally to induce the person to feel like a 
pawn in one situation and like an origin in another, 


when interrupted, d) completed the or igin model 
more elegantly, and e) recalled the nonsense name 

given the origin model more frequently a month after 
the experiment. Apparently, the origin- pawn dimen- 
Sion can be manipulated in а way that has Strong ef- 
fects on behavior. The Rotter I- 
istered to these subjects also, 


and n Achievement in Schools, and 
on n Achievement, I will report the data ыша 
n Achievement and Speculate about the O- m. 

The connection is closer than may be app 
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the achievement motivated person likes to take per- 
sonal responsibility for his actions. He likes to be 
ina position to act like an origin and avoids situa- 
tions where he is a pawn. 


Let us shift our emphasis, then, to the measure- 
ment of n Achievement in culturally disadvantaged 
children. First, let us be very clear about one 
thing. The word ''achievement'' is a loaded term, 
especially in educational circles. А culture such 
as ours, that has been nurtured on Horatio Alger 
stories and McGuffey readers, tends to perceive 
achievement motivation as an unmitigated virtue if 
not a blessing. If we pull back from this conception 
and ask what aspect of achievement motivation we 
can actually measure, we may be disappointed. The 
measure to be used was developed about fifteen 
years ago by McClelland and his associates at Wes- 
leyan University, and the subsequent years of re- 
search have told us much about the characteristics 
of people who score high on the measure. The most 
appropriate single word to describe the person with 
Strong achievement motivation as measured by this 
technique is that he is an ‘‘entrepreneur,’’ typically 
following a business career. Topcreative scientists 
are not particularly high on n Achievement, nor are 
academicians. If you want to succeed in business, 
however, you'd better have high n Achievement. 


The achievement syndrome has three major as- 
pects: 1) personal responsibility, 2) risk-taking 
strategies, and 3) the use of feedback. The entre- 
preneurial spirit of achievement emphasizes se lf- 
reliance, the taking of calculated risks, careful 
planning and checking of progress with constant em- 
phasis on the skillful use of one’s abilities. These, 
then, are the dependent variables that are related 
to the measure of achievement motivation. 


What is the measure itself? Actually, even after 
years of research the original form of the measure 
has not been improved very much and is essentially 
avery crude measure, Originally intended asa 
measure of individual differences, it yields at best 
a very rough ordinal scale that is most useful when 
collapsed into a high-low dichotomy, or at most a 
high- medium-low trichotomy. 


Before discussing the weaknesses of the measure, 
let us sketch the procedure for obtaining n Achieve- 
ment scores. The technique involves collecting 
thought samples from subjects under standardized 
conditions by showing a group of subjects several 
pictures, one by one, for a short period of time and 
asking them to write a creative story about each in 
turn. The stories are content analyzed by trained 
coders for instances of Achieve ment Imagery. 
Stories in which one of the characters is concerned 
over obtaining success in competition with a recog- 
mae standard of excellence are said to contain 
ae кез л The construct validity of the 
basa ai ested by years of research although 

re-test reliabilities are often lower than valid- 


ity coeffici 
jos. ficients (for reasons that we cannot take up 


ie ees concern to us is the question: Does 

of mote have any utility for the measurement 

ross vation in educational Settings and specifically 
culturally disadvantaged areas? Most of the early 


research results were disappointing in this regard. 
Apparently, n Achievement scores are not very 
highly correlated with school grades or standard- 
ized achievement test scores in high- school and col- 
lege students. As a result, educational interest in 
the measure lagged after an early flirtation, but it 
has recently been revived by the evidence that it is 
possible to change the motive by training. Before 
we get dazzled by the thought of changing motives in 
education, we should try to understand the relation- 
ship between n Achievement and school behavior. I 
think there are good reasons to believe that the 
measure has been misapplied but that it, oraderiv- 
ative of it measuring the O-P variable, may be very 
valuable for education. 


This misapplication of the n Achievement meas- 
ure stems from the two most basic levels possible: 
a lack of understanding of the theoretical basis for 
the measure, and lack of clarity about its scaling 
properties. 


The problem of understanding the theoretical ba- 
sis for the n Achievement measure is formidable. 
Exactly why there should be a relationship between 
the content of stories written by a subject in a one- 
half hour period and many other manifestations of 
his behavior, particularly school behavior, is still 
a problem shrouded in mystery. Iwill spare you 
a discussion of several explanatory attempts that 
have been made and present one based on the notion 
that the measure gives a sample of thoughts of the 
subject. 


Let us be very simple-minded and conceive of 
thoughts as a type of response, arbitrarily remove 
them from the ghostlike realmof the mind, and rele- 
gate them to the more prosaic realm of the physical 
world of behavioral events. After all, then Achieve- 
ment measure is actually based on observable re- 
sponses, not on pure thoughts. From this point of 
view, thought samples are a measure of one domain 
of responses that we wish to correlate with other 
domains. Notice in passing that we have made no 
assumptions about a causal chain, that thoughts are 
antecedent to or ‘‘cause’’ behavior. We havea habit 
of thinking that thoughts precede actions but there 
is very little evidence to substantiate it, and in the 
realm of achievement motivation, there is some evi- 
dence that the behavior may precede the th oughts 
(McClelland, 1966). 


The definition of what constitutes an achievement 
thought is critical in predicting what behaviors will 
be associated with such thoughts. The scoring тап- 
ual (Atkinson, 1958) identifies achievement thoughts 
by the term Achievement Imagery and says they can 
be seen in a story when one of the characters shows 
concern over competition with a standard of exc e l- 
lence. The character can show competition when 
any concrete goal is at stake, such as winning a 
race, getting a good grade, becoming a success in 
a career, building a better mousetrap, and so on. 
The non-specificity of the goal of achievement be- 
havior leads us to the scaling problem mentioned 
earlier to which we shall return, but let us pursue 
a little further the theoretical basis for the measure. 


Thought -sampling is based on an assumed corre- 
lation between thoughts and more concrete behavior. 


wu 
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The advantage of using a sample of thoughts rather 
than some other type of behavior is this: of all be- 
haviors, thoughts are least subject to inhibition, If 
we seek a measure of what a person wantstodo, we 
need to rid it of restrictions as to what he can do, 
For this and other reasons, thought-samples stand 
a chance of giving us a better picture of a person’s 
generalized motives. 


In attempting to measure what a person wants to 
do, we must follow quite a different road from that 
taken in measuring what a person is able to do. Put 
in another way, the measurement of motivation is 
quite different from the measurement of intelligence, 
ability, or even attitudes. We want to know what a 
person will do, not what he can do, although the lat- 
ter may form a limit in any particular circumstance. 
In measuring motivation, we sample his thoughts 
under neutral conditions: not when he is pressured 
and а pawn, but when he can originate some of his 
own behavior. By definition, in such a situation, we 
cannot have control of the kinds of res ponses ob- 
served. If we tighten control, we lose the ability to 
measure what he will do on his own and end up with 
a measure of what he can do when we pressure him. 


This paradoxical measurement situation is more 
or less unique and doesn't fit very nicely into some 
of the most basic conceptions of scaling. To make 
this a little clearer, think of Skinner's distinction 
between operants and respondents. Roughly, a re- 
Spondent is a response that we can elicit directly by 
manipulation of a Specific stimulus. We more or 
less directly produce a response in the organism by 
external manipulation. Salivary, pupillary, and eye- 
lid responses are examples. An operant is quite 
different, Although Skinner assumes that this type 
of response is probably elicited by some stimulus, 
we don’t know and may never be able to know what 
it is, so we cannot control the response through the 
eliciting stimulus, Ignoring control by reinforce- 
ment for the time being, we measure an operant by 


observing its spontaneous emissionunder conditions 
of relative freedom. 


Let us now conceive of thought sampling as meas- 
uring the rates of Spontaneously emitted thoughts, 
і. e., as the measurement of operants. Here we 
launch into new territory. While Skinner is primar- 
ily concerned with Shaping a specific o perant and 
converting uncontrolled behavior into behavior that 
is under the control of the experimenter, we are 
concerned with measuring a generalized class of op- 
erants in the realm of thoughts and relating this 
measure to other more concrete behavior patterns. 


Sitivity along an intensity 
ry to illustrate this: Pictures 
ment | samples from which n Achieve- 
ras eos are derived show a variety of charac- 

> for Instance, a high-school student, two inven- 
Person (AY. businessmen, etc. In what sense cana 
Ventorg ) who writes achievement stories about in- 
high sco, doctors, and businessmen and attains a 
than е be said to be h igher on n Achievement 

another Person (B) who writes an achievement 


imension, [et me t 
used to elicit thought 


Story about a high- school boy but not about inventors, 
doctors, and businessmen and receives a relatively 
low score? If we observe many ty pes of behavior in 
many situations, we may assume that person A will 
Show more general achievement behavior than per- 
son B. But if we measure only school behavior, the 
reverse may be true. Inan unpublished study, we 
have demonstrated that a six picture measure of n 
Achievement derived from a standard set does not 
meet the criterion set by the Guttman technique for 
unidimensional scalability (Stouffer, Guttman, Such- 
man, Lazarfeld, Starr, and Clausen, 1947). 


Does this mean that the measure is simply unre- 
liable? I think not, although it does mean that it is 
not a unidimensional measure of the intensity of a 
motive to do a very specific thing and, therefore, 
should not be expected to relate to or predict rather 
Specific behavior, such as that expected on a stand- 
ardized achievement test taken under conditions that 
produce respondent, or at least discriminated oper- 
ant, rather than Spontaneously emitted Operant type 
behavior. It would be nice to be able to measure 
motivation along an intensity scale, and most of mo- 
tivation theory influences us to think of motivation 
in terms of intensity concepts such as need and drive, 
If we think back for a moment to the discussion 
of the Origin- Pawn dimension, however, we can see 
that a generalized measure of what a person will do 
when not forced may come closer to our concept of 
motivation than an intensity measure of what a per- 
Son can do when pressured, 


In the book presenting the original development 
of the n Achievement measure, McClelland, Atkin- 
Son, Clark, and Lowell (1953) briefly mentioned that 
the technique measured the extensity of Achievement 
thoughts in a broad realm rather than the intensity 
of such thoughts. This mention has apparently been 
lost in the flood of research, most of which was not 
careful to note the the distinction. 


As suggested above, a careful 
Sic process involved in the 
gives some insight into the 
п Achievement and academi 
us more. I think it gives 


analysis of the ba- 
measurement technique 
low correlat ions between 


estir 
that n Achievement Scores will be related in e F 
more general response patterns Since the measur 
taps thoughts about achie НЧ 


often than not. In view of the an. 
pected that the measure of n A 


(McClelland, 1961) 
individuals, than in 
such as school achievement or 
One example may be cited, 19 
ports that of the orig te ee 
> Sted at 

D Achieve. 


had high n Achievement 


Scores ag Sophomores i 
in 


36 RESEARCH AND DEVELOPMENT TOWARD THE IMPROVEMENT OF EDUCATION 


college. Only 21 percent of those classified as non- 
entrepreneurs had high n Achievement. These re- 

sults indicate that n Achievement predicts career 

patterns over a crucial period of life when c areer 

choices are being made. 


When it comes to measuring motives in culturally 
disadvantaged children, we have a lot to learn. The 
data that I have to report results from a study that 
was initially intended to answer some of the most 
rudimentary practical questions such as ‘Сап 
thought samples be collected from this population, 
i. e., can they produce Scorable stories under 
standardized conditions?" If it turns out that the 
thought samples are scorable, we can ask whether 
the n Achievement score is valid for this population. 
Is the score, for instance, related to risk-taking 
Strategies in this population as it is in middle-class 
high-school and college students? Since we were 
interested in beginning to probe the Origin-Pawn 
dimension in this population, we Sought an existant 
measure that would give an estimate of the children’s 
relative feelings of powerlessness. In short ‚ ме 
tested the feasibility of collecting thought- samples 
and found fifth, sixth, and seventh grade disadvan- 
taged children quite capable of producing scorable 
stories. We devised a risk-taking situation that had 
the bonus of providing us with a measure of school 
skills, and we used a measure of powerlessness, 


The subjects of the study were ninety-four Negro 
children in two fifth and seventh grade classrooms 
in one large city (sixty girls and thirty-four boys), 
and 120 Negro children from several sixth grade 
classrooms in another large city. Both sa mples 
were taken from school areas known to be the most 
culturally disadvantaged in each city. However, 
since we were primarily interested in testing the 
feasibility and validity of the motive measure, we 
did not sample meticulously, nor did we carefully 
assess the comparability of the areas, For this 


reason, comparisons between the two samples are 
dangerous, 


THE MEASURE 


MENT OF ACHIEVE 
MOTIVATION aia 


ded a Six-page form withad 


equally spaced on еа; 

their stories, ch page to help them complete 
‘Two men are 
Children were 
Stories, 


These stories Were content anal: 
lyzed by the pro- 
Cedures des, by McClelland, et al. (1953) by 


Interscorer reliability was 
Percent agreement between scorers, 


THE MEASURE OF POWERLESSNESS 


Cribed 
two independent Scorers. 
approximately 90 


ever help any to think what you will be when you 
grow ир?” 


RISK- TAKING BEHAVIOR 


In a game situation in which the person can 
choose to shoot for a fixed target from varying dis- 
tances of increasing difficulty with increasing re- 
ward attendant on Success, a group of high-achieve- 
ment persons will tend to take more shots in the 
middle range, avoiding both easy and extremely 
hard shots. A similar group of low - achievement 
persons will take more shots in the easy and ex- 
tremely hard positions (cf. , Atkinson, Bastian, 
Earl, and Litwin, 1960). The range of risk which 
may be considered a moderate risk is dependent up- 
on the skill of the individual and is evidently chosen 
in terms of where he thinks he has a mode rate 
chance of success (cf., deCharms and Davé, 1965). 


We rejected the simple game situation most often 
used. Instead, it was decided to attempt to develop 
a new situation completely analogous to the game 
utilized in our earlier work but involving the typical 
school behavior of children, i. e. spelling and arith- 
metic skills. 


SPELLING AND ARITHMETIC SKILLS 


Each of these measures was comprised of sixty 
items divided into six levels of diff iculty of ten 
items each, Spelling words were taken from stand- 
ard lists calibrated for difficulty, and arithmetic 
problems were taken from a series of arithmetic 
textbooks used in the schools fromearly elementary- 
grade level to junior-high- school level, Extensive 
pretesting was used to assess empirically the valid- 
ity of the various levels, and to assure a highdegree 
of success for all children in the easiest level, and 
increasing difficulty up to a level where little suc- 
cess could be expected from any child. 


These measures were administered by standard 
classroom procedures, The items were arranged 
in blocks of six, each block progressing from the 
easiest to hardest level of difficulty inorder, though 
this was not made obvious to the children. It was 
stressed that this was not a test in the sense that the 
Children would receive grades, but that they them- 
Selves would learn from this about their own skill. 


SPELLING AND ARITHMETIC RISK- TAKING TASK 
After the 
with each chil 
sixty items he had attem: 
completed correctly at e. 


The child was told that he ул 
in which he с " 
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Before he was presented with each item, he was to 
choose the level of difficulty which he would like to 
attempt on that trial. The object was to attain 
**points, " the significance of which was left unex- 
plained, as in a game. The points that could be won 
by correctly completing the item increased with the 
level of difficulty of the item chosen. These points 
were established for each child separately, accord- 
ing to the empirically established level of difficulty 
for him. Thus, a skillful child who chose level five 
(a difficult level), and who had done four out of ten 
items correctly at that level in the skill-task (em- 
pirical probability of success equals . 40), stood to 
gain thirty-eight points if correct. A less skillful 
child choosing the same level five may have done 
only one out of ten items correctly at that level in 
the skill-task (empirical probability of success 
equals . 10). Such a child could win 150 points if 
Correct at levelfive. (This procedure, developed 
by deCharms and Davé (1965), equates subjects as to 
skill and emphasizes choices in terms of the child’s 
own chances of success, ) 


Each time the child chose a levelfor the next item 
(of ten spelling and ten arithmetic items), he was 
asked to consult his own paper from the skill-task 
and report how many he could expect to get right 
out of ten (his probability of success at that level). 
He was then asked to consult the following table and 
report how many points he could win. The point 
Scale was constructed so that the number correct 
multiplied by the number of points always approxi- 
mated a constant (1 50) as closely as possible without 
using decimals, The expected value of ten trials at 
any level, therefore, equals 150. 


Number you got right 


Out of ten оп skill-task 10 9868765 4 3210 


Points you get if right 15 17 19 22 25 30 38 50 75 150 300 
By such means the experimenter 


made every ef- 
fort to insure that the child thought 


of his choices 


signed in accordance with his own demonstrat ed 
skill, 


Each child made ten choices and attempted ten 
Spelling words, recording his accrued points after 


each trial. Following this, each child did the same 
with arithemtic items, 


RESULTS 


Table 1 presents the mean skill scores on arith- 
metic and spelling tasks by sex and level of achieve- 
ment motivation. You will note that the means for 
the high п Achievement subjects are higher in all 
Cases than the means for low n Achievement sub- 
Jects. The differences tested by analysis of vari- 
ance are Significant in every case except for arith- 
metic for girls, There is clearly a relationship over- 

etween n Achievement level and school skills 
“4 Sured by our test. Therelat ionship is 
йге SER for boys than for girls, but all differences 
аен “rt à fact that is attributable to the low var- 
Signeq the scores, The 60-item task was de- 
80 tha 


" ar 
t the majority of the s ubjects would motivation will more often Choose Spelling words ог 


Success on almost all of the items at the easiest 

level and fail on almost all at the most difficult lev- 
el. Asa result, the range of probable scores is 

much smaller than the total range of sixty, and the 
variances are Quite small. In addition, these re- 
sults come from a larger analysis where variance 

attributable to other factors, such as grade level, 
was extracted. 


Table 2 presents data Írom the same dependent 
variables, Spelling and arithmetic Skills, but this 
time showing the relationship with the I-E scale. 


а questionnaire, and the differenceis interesting the- 
oretically. Second, a recent review of the literature 
9f n Achievement indicates à similar trend toward 
an interaction between Sex and ty 


Pp. 297-298), Klinger makes no attem 
this interaction 


In view of our analysis of thou 
or less “free 


es. І cannot pretend to 
hould be so 


in the achievement area, 


The results from the risk-takin, dat: 
sented in Tables 3 and 4, and та b mid 


land 2. The means in the tables ne. 
planation. The hypothesis de 
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TABLE 1 


MEAN SPELLING AND ARITHMETIC Si 


CORES (OF A POSSIBLE 60) BY SEX AND LEVEL OF 
n ACHIEVEMENT 


Boys Girls 
es ee лкы 


Math** Spelling** Total Math Spelling * Total 

High N 46 46 60 60 
n Achievement x 21.7 28.2 24.9 22.5 31.0 26.8 
Low N 51 51 63 63 
n Achievement x 18.5 21.7 20.1 19.6 26.5 23.1 
Total 20.0 24.8 22.4 21.0 28.7 24.9 

^f S08 
Np xS DT 
TABLE 2 


MEAN SPELLING AND ARITHMETIC SCORES (OF A POSSIBLE 60) BY SEX AND I-E LEVEL 
(POWERLESSNESS) 


Boys Girls 
PER Rp ee ee 
Math Spelling Total Math* Spelling* Total 
High N 46 46 69 69 
EE x 20.9 25.2 23.1 22.6 29.8 26.2 
id N 51 51 54 54 
x 19. 3 24.4 21.9 19.0 21.2 23.1 
Total 
20.0 24.8 22.4 21.0 28.7 24.9 
*p. «.05 
arithmetic probl i 
ability levels bon ep e middle ranges of prob- hypothesis then predicts that Subjects with high n á 
Probability, while Subjecte wit to 70 percent Achievement will ha ve smaller average deviation 
Will not show this tendency t ith low n Achievement Scores than subjects with low n Achievement. Ta- 
may, in fact, avoid the middle 5.87681 P аи апа bles 3 and 4 present the means of the average devi- 
о: i n i 
aoe extremes either around 90 percent or [ed Buen re each group. We shall call this 
Score. 
In order to extr 
T act 
test this hypothesis Slate o Írom the data to Table 3 gives Strong support for the hypothesized 
dure developed deCharms E hie used a proce- relationship between moderate risk-taking and high 
average deviation of the ten prob, avé (1965). The п Achievement under all conditions, i.e., for both 
aun 18 measured around the yee for до and girls апа when the task involves either 
ices for all sup mean Spelling or arithmeti ble 4 indicate 
Viation ing jects. A large aver «3 that Ф іс problems. Table 4 indicates 
range f thoes that the individual Subject’ mede DES 


cale is not related to risk-taking under 


Schoices any conditions, 


A 
ge. The with a Ret Figures 1 ang 2 may make the results 
п Achievement Clearer, F igure 1 combines 
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ТАВІЕ 3 


MEAN RISK- TAKING SCORES BY SEX AND LEVEL OF n ACHIEVEMENT 
(LOW MEAN INDICATES MODERATE RISK) 


Boys Girls 
Panic DR dii ee ne a c 
Math* Spelling*** Total Math* Spelling*** Total 
High N 46 46 60 60 
А n Achievement х 19. 2 24. 2 21.7 22.0 23.3 22.6 
Low N 51 51 63 63 
n Achievement X 23.8 29.1 26.4 24.4 27.9 26.1 
Total 21.6 26.8 24.2 23.2 25.6 24.4 
* o S5 
***p. «.005 
TABLE 4 
MEAN RISK-TAKING SCORES By SEX AND LEVEL I-E (POWERLESSNESS) 
(LOW MEAN INDICATES MODERATE RISK) * 
t 
Boys Girls 
асос сИ асса ЕЕЕ, 
Math Spelling Total Math Spelling Total 
High N 46 46 69 69 
I-E x 21.7 27.5 24.6 22.7 25.3 24.0 
Low N 51 51 54 54 
I-E x 21.5 26.2 24.4 23.8 26.0 24.9 
Total 21.6 26.8 24.2 23.2 25.6 24.4 
*No significant differences 
boys and girls ang Shows a hi i 
gh peak in the mid the possibility that risk-taking on as 
ranges of probabilities of Success for the high n Such as spelling or arithmetic is moree ie 3 s fina 
Achievement Subjects on arithmetic problems. The function of n Achievement than on the game a ү 
curve for the low n Achievement Subjects shows an tasks used in other studies. Tt is also possib] ps 
interesting Propensity to take extremely speculative the effect of achievement motivation ma be Pin 
risks. For the low n Achievement Subjects 58 per- clearly evident in the culturally disadvant. Oe 
cent of the choices are in the range of 0 to 40 per- ronment than in the highly competitive iss anys 
cent Probability of success. i 


Figure 2 А 
the high n „не the results for Spelling. Here 


5 п Achievement cur in the 
middle, Bud e rve shows a hump in 


a DISCUSSION 
phows a trough in the middle ranges and is higher at 


Temes as Predicted by the hypothesis. 


All of the results reporte 
sho rese results with : 


d fro i 
arithmetic and spelling tasks derived from two different варь ы рау were 
gics ben ёагег differentiation in risk-taking strate- bined for simplicity of Presentation. рт com- 
than mos, C? high and low n Achievement subjects ples are investigated Separately, the fi di, '$ Sami- 
Ost previous Studies in this area, and suggest ported replicate in ndi 


almost every case. ugs re. 
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Figure 1. Percentage of tasks chosen at each level 
of probability of success on the arithmetic risk- 
taking task by subjects above and below the median 
n Achievement score. 
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Replication of the data on spelling and arithmetic 
skills is especially important since the tests from 
which these scales derived were intended originally 
merely to give preliminary data for the risk-taking 
task. The fact 
ment are significant at all is surprising and even 
embarrassing since the major thrust of our theoret- 


ical argument predicts that the n Achievement meas- 
ure should be 


change over ti 
as these arithmetic an 


S quite 
weenn 
Although 


Some indicati, 
tivation is a Specific area, 


A Probably the most interest: 
liminary f 


r ing result of these pre- 
orays into the study 


of motivation in cul- 


NEM oo o. 


" 


Figure 2. Percentage of tasks chosen at each level 


of probability of success on the spelling rish-taking 
task by subjects above and below the median n 
Achievement score. 


% Chosen 


Probability of Success 


turally disadvantaged children is the evidence that 
the thought sample measures are more effective 
with boys than girls, and with the questionnaire 
measure the reverse is true. If this result stands 
up on replication, it may lead to further insights 

about thought sampling. 


It should be clear by now that the I-E scale used 
here was not designed to measure the origin- pawn 
dimension. It is our hope that a measure of this di- 


mension can be devised based on thought sampling. 
To be valid, the m 


perimenter as on a question- 
- We areat present beginning 


under way, the aims of i 
Achievement to change i 11 i- 
tudinal study, т сш 


nd measure 
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the origin- pawn dimension, and ultimately to develop 
training techniques for teachers and pupils similar 
to those used in achievement motivation training 
courses with businessmen (McClelland, 1965). 
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TUNI PART Il 


ium Subject Matter, Content and Sequence 


CURRICULUM improvement in the various disciplines has been ham- 
pered by lack of adequate research paradigms to determine the effectiveness of 
various instructional programs, by lack of relevant learning theory, and by inade- 
quate conceptualizations for determining the content to be learned. The structure 
of the discipline itself, as formulated by one or more scholars, has provided 
much of the basis for recent curriculum development. E. C. Begle in "Curricu- 
lum Research in Mathematics" cites the extensive work of the School Mathematics 
Study Group in the development and evaluation of a mathematics curriculum. The 
structure of mathematics has proven to be a powerful organizer for the mathe- 
matics curriculum at the university, high school, junior high school, and now at 
the elementary school levels. The point of view adopted is that in learning mathe- 
matics each Pupil is to build his own structure of mathematics. Such a structure 
starts with a small number of concepts and eventually incorporates strategies of 
problem solving, Begle argues that the necessary framework would be charac- 
terized by three aspects: "1) the mathematical content of the curriculum, 2) a 
theory of learning of the content, and 3) a theory of the application of the con- 
tent to problem solving.” Begle reports that some of the results obtained by the 


School Mathematics Study Group are being investigated in a National Longitudinal 
Study of Mathematical Abilities. 


Similar problem 
mon to science. Arthur Live 
science) as the basis for curri 
(PSSC in physics, BSCS in bio } 
Union of Scientists, 
lem of curriculum d 
Curriculum in scienc 
high school level, 
While these curri 


educators, and psychologists wor 


king together on the prob- 
esign and evaluation. 


The evolution of the so-called new 


School levels. 


Curriculum development in English 
discipline, Robert Pooley discusses 
nales and materials 
guage, and in the emerging gram 
research underway in the various 
given to the structural and transfor 


Reading ability is of 


is also dependent on the struc- 
various eff 
in such areas as 


ture of that orts at developing cur- 
riculum ratio 


literature, composition, 
mars of the English language. 


areas are reported with parti 
mational grammars, 
fundamental concern 


s Barrett in 
"whether varia- 


typically found in classroom material.” Extensive details on proc 
sults are reported. 


The contribution of 
in mathematics 


=== Curriculum Research 
in Mathematics 


I SHALL interpret “curriculum” in 
the above title very broadly, since I believe that re- 
search in mathematics education must involve re- 
Search on the learning of mathematics, the teaching 
of mathematics, problem solving, etc. , as well as 
the syllabus, and that exclusive concentration on any 
one aspect of this complex is not fruitful 


Search in mathematics education. Weaver (1957- 
lists covering researchon 


; the results are 


too special or incomplete to be of any wide use. 


I 
sean take much reflection to see w hy this 
ee 22 the case, Research in education in gen- 
Mich more die ee education in Particular, is 
» 8 
ке chemise than research in Say, psychol- 


Е Fo i 
variables that should be ‘tak i, eam pier of 


at least 


=== E, G. Begle 


Stanford University 
Stanford, California 


jects, by a wide variety of cognitive abilities and 
cognitive styles, by attitudes toward mathematics, 
and, as we well know, by socio-economic and demo- 
graphic variables. Similarly the cognitive and af- 
fective state of the teacher may well affect the stu- 
dent's learning of mathematics. 


In order to isolate and compare two different cur- 
riculum treatments, for example, itis necessary to 
take all these variables into account so that the ef- 
fects due to these variables will not be confounded 
with those due to the curriculum treatments. To do 
this generally requires either a large experiment or 
a large number of carefully correlated small ones. 


Until very recently, it has been almost impossi- 
ble to meet this requirement. Experiments in math- 
ematics education have been carried out by individ- 
uals with limited resources. The experiments of ne- 
cessity were small in scale, involved relatively 
small numbers of Subjects, and had a short time 
Span. Only very recently has financial support be- 
come available which makes possible some of the 
Studies and experiments necessary for significant 
improvement in mathematics education. 


A large scale investi; 
by the School Mathemati 


hen, 1965). 


In this study three 
grades 4, 7, and 10 in September 


groups of students enterin g 
» 1962, totaling 
beenfollowed for 
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The primary purpose of this study isto ascertain 
the effects of the new mathematics curricula inor- 
der to provide guidance for further curriculum de- 
velopment. 


It was necessary, of course, to measure not on- 
ly the level of mathematics achievement of each stu- 
dent at the beginning of the study, but also as many 
as possible of the other variables which might affect 
the results. We have therefore obtained measures 
of a considerable number of Cognitive variables, 
selected by our psychological consultants, for their 
probable relevance to mathematics achievement. An 
extensive attitude inventory was administered three 
times during the study, since we want to know how 
attitudes affect achievement and also how achieve- 
ment affects attitudes. We have some information 
about the socio-economic status of the students as 
well as information about the schools. Most of the 
teachers of the students included in the study have 
provided us with extensive information about their 
training and experience, and more than half of them 
have also provided information about their attitudes 
toward mathematics, teaching, etc. 


When we analyze this enormous mass of informa- 
tion, we will not only find out what we want to know 
about the differential effects on mathematics achieve- 
ment of modern curricula as compared with conven- 
tional ones, but we will also obtain as a by- product, 
information on the effects on mathematics achieve- 
ment of the other variables we have measured. 


This kind of information should prove valuable 
both to the design of future experiments and in the 
interpretation of past results. Hopefully we will 
find that many of these variables have little or no 
effect on mathematics achievement. When this is 
the case, future experiments ir 


Work. 


THE MATHEMATIC 


AL CONTENT OF THE 
CURRIC ULUM 


it of course willbe concerned with 
tent of the curriculum. Before 
rn to what may see m at 
Iam sure that many of our 
in schoo look on our new curricula as a revolution 
In a sense this is correct, 
aspect of a revolution in 


е hematical coni 
lscussing this, I will tu 


but it is merely a small 


mathematics itself which has been going onfora 
century and a half. 


This revolution stems from the work of Abel and 
Galois. When Abel demonstrated the nonexistence 
of an algebraic formula for the solution of the quin- 
tic equation and when Galois applied his methods to 
settle the general problem, the direction of the de- 
velopment of mathematics was changed forever. Ob- 
viously they did not obtain their results bytime- 
honored computational procedures. Instead they dem- 
onstrated that an examination of the over-all struc- 
ture of a mathematical System, in contrast to com- 
putations with the individual elements of the system, 
Was a very powerful mathematical tooland could 
lead to solutions of problems not otherwise Solvable, 


Although the revolution dates back to the previous 
century, it gathered momentum Slowly, and it was 
only in the present century that its effects became 
overwhelming. The tremendous flowering of modern 
algebra and of functional analysis which start ed 
early in this century could not have taken place with- 
out the ideas about structure first pointed out by 
Abel and Galois. 


During the 1930's these ideas began to be applied 
in the graduate curriculum, and it became clear that 
these ideas, already demonstrated to be powerful 
and indispensable in mathematical research were 


tion at this level. During the next two decadesa 
start was made on 
curriculum in the 


mathematicians joined with 


attempt to improve the 
secondary-school curriculum, their first inclina- 


tion was to apply to the subject matt! 


curriculum, the reform mov 


mentary- schoo] Curriculum in he 
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On the other hand, the new curriculum differs 
radically from the old in that it includes the struc- 
ture of the common mathematical Systems, the 
basic mathematical Concepts and their interrelation- 
Ships, as well as the basic mathematical facts and 
techniques. 


Before discussing the implications of this new 
aspect of the curriculum for our theoretical frame- 
work for mathematics education, it is well to point 
out that while few new topics have been incorporat- 
ed in the curriculum So far, there will undoubtedly 
be new ones to be added from time to time. The na- 
ture of our society at present is such that it would 
be wise to include as soon as possible, and for all 
students, some of the basic ideas of probability and 
Statistics as well as some of the mathe matical ideas 
relevant to high speed computers. Incorporation of 
any such new topic into the curriculum will require 
research to determine the Prior understandings 
needed by the student, the most efficient location of 
the topic in the sequence, etc. Wecan certainly 


look forward to a continuing need for this kind ofre- 
search. 


Returning now to the Curriculum, we see that 
our aim in mathematics education is to help each 
Student to build in his mind a Conceptual structure. 
This structure, for primary-school children, starts 
with a small number of concepts, abstracted di- 
rectly from the real world, about number and geo- 
metric configurations. As the child progresses 
through school, these concepts are refined and 
sharpened, and the relationships between them be- 
came increasingly complex. At some stage, new 
concepts are met which come not f ro m the real 
world but rather from previously learned concepts. 
The concept of irrational number is a case at point. 
Among these concepts there are, of course, those con- 


cerned with computation. For example, we do not 
ask our students to memor 


tion table, Rather, we let 


Naturally, the order in which mathematical con- 
cepts are learned is subject to some variation, and 
h time could be devoted 


A THEORY OF THE LE 
CONTENT ARNING OF THE 


Concepts, but interest is usually focused onthe 
Strategies used by the subjects in trying to guess 
which combinations of color, sizes, shapes, ete., 
the experimenter has chosen. Neither the concepts 
nor the strategies have much re levance to class- 
room learning. 


An interesting by-product, however, of these 
studies is that the availability of verbal labels for a 
concept plays an important role (Kuenne, 1946) and 
will have to be taken into account in any theory of 
the learning of mathematical concepts. 


Some of the investigations of *discovery'' teach- 
ing in this country have involved the learni ng of 
some non-trivial mathematical concepts. Unfortu- 
nately, the results of these studies present a con- 
fused picture. For a review of the present state of 
affairs, see Cronbach (1966). 


Outside the U.S., investigations of concept learn- 
ing have been dominated by the work of Piaget. „Не 
deserves great credit, of course, for investigating 
real mathematical concepts and how children come 
to acquire them. Unfortunately, he generally c on- 
fines himself to the initial concepts of any branch 
of mathematics and has little to tell us about the 
learning of the later and more refined concepts or 
the learning of relationships between concepts. 


A more fundamental difficul 
findings in mathematics educat 
that, when he questions a 
understands a concept, he is very careful never to 
provide the child with any subsidiary concepts which 
might help him. In fact, he seems to be investi- 
gating what children can learn without the benefit of 
education. There are, however, some indications 
(Kohnstamm, 1965) that the picture is quite differ- 
ent if children are provided with some assistance in 
the form of subsidiary concepts. 


ty inapplying Piaget's 
ion lies inthe fact 
child to see how well he 


In short, the task of building a theory of the learn- 
ing of mathematical concepts still lies before us. 
Some ideas about what will have to be inclu ded in 
such a theory can be obtained from the sources just 
mentioned. It should also be mentioned that some 
very interesting work, with which we are just be- 
ginning to become acquainted, is being done in Rus- 
Sia, and this work may provide us with further clues. 


In order to construct a suitable theory ofthe 
learning of mathematical conce pts, andsuch a 
theory should be quantitative rather than purely qual- 
itative, it will be necessary toconstruct instru- 
ments which measure how wella student under- 
stands a particular concept. 


This will not be an eas: 
observable, It will be ne 


methods to assess а student's und 
ticular concept. Per. 


y task. Concepts are not 
cessary to use indirect 
erstanding of a par- 
арѕ we can obtain Some guid- 
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in the development of instruments for measuring un- 
derstanding, although the final in struments will 
need to appear in a more standardized format. It is 
worth observing that Brownell (1963) also has found 
working with individual children a valuable pro- 
cedure. 


THE APPLICATION OF THE CONTENT TO 
PROBLEM SOLVING 


A final aspect of our theoretical framework for 
mathematics education is that of problem solving. 
It would be futile to provide our students with a 
thorough understanding of mathematics if they were 
not able to use this understanding to solve problems. 
What we need therefore is a better understanding of 
how we can help our students to developuseful strat- 
egies for solving problems. 


This seems to call for a considerable amount of 
exploratory work. We do have already, however, а 
variety of problem- solving Strategies that can be 
investigated. Polya (1945) for example, has written 
extensively on this topic. In another direction, 
Newell, Shaw, and Simon (1959) have investigated, 
by means of computer simulation, the power of cer- 
tain specific Strategies. These Strategies had been 
identified as ones actually used by at least some 
Students, and a computer program then verified 
that the Strategies were indeed effective. Instill 
another direction is the very interesting work by 
Crutchfield and Covington (Crutchtield, 1966). Using 
comic-book format for their materials, they were 
able to train their subjects to examine the relation- 
ships between the various aspects of a problem sit- 
uation, and the implications of these relationships, 
fora sufficiently long time and in sufficient de pth 
to arrive ata solution. Their problems were not 


mathematical, but their procedures may well have 
implications for mathematics education, 


The Russian literature, mentioned above, also 
has suggestions of problem- solving strategies which 
are worth investigating. 


There are still two other sources which may pro- 
vide us with raw material for investigation of prob- 
lem solving. The Strategies used by Newell, Shaw, 
and Simon were obtained by having a number of stu- 
dents solve problems aloud during individual inter- 
Views, This procedure may well prove very fruit- 
ful, especially at the elementary school level, where 
We seem to know less than at thehigher levels. 
Similarly, classroom observations of teacher 5 
teaching problem solving and of their techniques for 


helping Students to solve problems may pay good 
dividends, 


In short, we already know a number of problem- 
we ving Strategies and we have suggestions for lo- 
cating others, What we need to do now is to start a 


Systematic Study of techniques for teaching these 
Strategies to st 
use t 


© of mathematics a student needs before he 


This БОШ а Particular strategy and use iteffectively. 


Сап be usay UP again the need for instruments which 
i ied to measure understanding of mathemat- 
Pts, 


To summarize briefly, I have been arguing that 
substantial improvements in mathematics education 
will have to be based on more fundamental research 
than has been carried out so far. I have further ar- 
gued that the magnitude of the task is such that the 
efforts of a large number of research workers will 
be needed and that these efforts can be made mutu- 
ally reinforcing and contributive to substantial im- 
provement only if there is an overall theoretical 
framework which is generally accepted and which 
can be used not only to suggest individual research 
problems but also to relate individual findings ina 
meaningful pattern. 


Three aspects of such a theoretical framework 
were mentioned: the mathematical co ntent of the 
curriculum, a theory of the learning of the content, 
and a theory of the application of the content to prob- 
lem solving. The first is, except for minor varia- 
tions, dictated by the nature of mathematics itself. 
It differs from the curriculum content of a genera- 
tion ago by emphasizing the conceptual structure of 
mathematics as well as routine mathematical skills. 
This part of our theoretical framework, then, is gen- 
erally agreed to. The rest however, we will hav e 
to construct. We have a few bits and pieces to start 
with, but a great deal of empirical work will have 
to be done before even the outlines of these aspects 
of our general theory will emerge, 


Finally, I have argued that an essential tool that 
we must have to carry out our task isabattery of in- 
Struments which measure understanding of math es 
matical concepts, and hence that our fir st step 


should be that of starting to construct s uch instru- 
ments. 


CONCLUSION 


In conclusion, I Should like to note that any frame- 
work we may construct for mathematics education 


and not immediately general- 
t matter areas, In mathemat- 


of physical Situations, In Science hi 

y Owever, for ex- 
ample, many more different as cts i 
considered, e, g-, life, i^ > Kee Sad 
color, density, galaxies 
periodic table, 


€ remote Cur 
areas, such as social science or literat 


ample, the situation is probably 


Ticulum 
ure, for ex 
even more different 


REFERENCES 


Brown, K. E. Analysis of researchi 
of mathematics: 1955 and 1956. 
4. Washington, D. D,- U.S. Off 
tion, 1958. 


48 RESEARCH AND DEVELOPMENT TOWARD THE IMPROVEMENT OF EDUCATION 


Brown, K. E., and Kinsella, J. J. Analysis of re- 
search in the teaching of mathematics: 1957 
and 1958. Bulletin No. 8. Washington, D.C.: 
U.S. Office of Education, 1960. 


Brown, K. E., Kinsella, J. J., Fehr, H., Myers, 
S. , and Weaver, F. Analysis of research in 
the teaching of mathematics: 1959 and 1960. 
Bulletin No. 12. Washington, D. C.: U. S. 
Office of Education, 1963. 


Brown, К. E., and Abell, Т. L. Analysis of re- 
search in the teaching of mathematics. Bulle- 
tin No. 28. Washington, D. C.: U.S. Office 
of Education, 1965. 


Brownell, W. A. Arithmetical abstractions - prog- 
ress toward maturity of concepts under differ- 
ing programs of instruction. The Arithmetic 
Teacher, 1963, 10, 322-329. 


Cahen, L. S. An interim report on the national 
longitudinal study of mathematical abilities. 
The Mathematics Teacher, 1965, 58, 522-526. 


Cronbach, L. J. The logic of experiments on dis- 
covery. In L. S. Shulman and E. R. Keislar 
(Editors), Learning by discovery: A critical 


appraisal Chicago: Rand McNally, 1966. Pp. 
76-92. 


Crutchfield, R. S. Creative thinking in children: 
Its teaching and testing. In О. С. Brim,R.S. 
Crutchfield, and W. H. Holtzman (Editors), 
Intelligence: Perspectives 1965. New York: 
Harcourt, Brace and World, 1966. Pp. 33-64, 


Kohnstamm, G. A. Developmental psychology and 


the teaching of thought operations. Paedogo- 
gica Europaea, 1965, 1, 79-97. 


Kuenne, Margaret R. Experimental investigation of 


the relation of language to transposition behav- 
lor in young children, 


Journal of Experimen- 
tal Psychology, 1946, 36, 471-490. 


; and Simon, H. A. Re- 
rt on а general proble olving program. 


: Proceedings of the Internation- 


Information Proce si 
UNESCO House, 1959. а 


Polya, G. How to solve it. 
Princeton University 


Princeton, New Jersey: 
Press, 1945, 


Wallace, J. G. Concept growth and the education 
of the child: A survey of research on concep- 


tualization. Occasional Publication Series No. 


12. Slough, U. K.: National Foundation for 
Educational Research in England and Wales, 
1965. 


Weaver, J. F. Six years of researchonarithmetic 
instruction: 1951-1956. The Arithmetic 
Teacher, 1957, 4, 89-99. 


Weaver, J. Е. Researchonarithmetic instruction — 


1957. The Arithmetic Teacher, 1958, 5, 109- 
118. 


Weaver, J. F. Research onarithmetic instruction — 
1958. The Arithmetic Teacher, 1959,6, 121- 
132. 


Weaver, J. F. Researchonarithmetic instruction — 


1959. The Arithmetic Teacher, 1960, 7, 253- 
265. 


Weaver, J. F. Researchon ele mentary-school 
mathematics – 1960. The Arithmetic Teach- 
er, 1961, 8, 255-260. (a) 


Weaver, J. F. Research on elementary -school 
mathematics — 1960 (concluded). The Arith- 
metic Teacher, 1961, 8, 301-306. 


Weaver, J. F. Researchon ele mentary- school 


mathematics – 1961. The Arithmetic Teach- 
er, 1962, 9, 287-290. pape 


Weaver, J. F. Researchon eleme ntary-school 
mathematics - 1962. The Arithmetic Teach- 
er, 1963, 10, 297-300. 


Weaver, J. F. Research on elementar y-school 


mathematics – 1963. The Arithmetic Teach- 
er, 1964, 11, 273-275. 


Weaver, J. F. Addenda to the May, 1965 report on 
research in elementary school mathematics. 


The Arithmetic Teacher 1965, 12, 577-578. 
(a). 


Weaver, J. F. Research on ele mentary- school 


mathematics - 1964. The Arithmetic Teach- 
ег, 1965, 12, 382-387. (b). 


Weaver, J. F. Researchon mathematics education, 


grades K-8, for 1965. The Arithmetic Teach- 
Er, 1966, 13, 414-427. 


' Curriculum Research 
in Science 


Science teaching has been possible because 1 
Sums of money have been available from the Nation- 
al Science Foundation to finance the activities. The 

early curriculum reforms came at the high school 
level and these are now well established in school 
Systems throughout the Country. Because they are 

well known I will not elaborate on them, but it is im- 
Portant to review them briefly because they have A 

in large measure, set the pattern for sc ience cur- 
riculum research in the pre- high school grades. 


HIGH SCHOOL CURRICULUM RESEARCH 


These high-school curriculum projects did sev- 
eral important things. Inthe first place they 
Stripped away the authoritarian discussions of mod- 
ern technological advances which had been tacked 

О Science courses that were tw enty years ог 
teste ош of date. They adopted the philosophy that 
al Ching Science was much more than teaching a cat- 

O8 Of facts, Rather, it should be teaching facts 
Oncepts and developing at the same 
tanding of and an ability to engage 
€ Scientific enterprise. 


together with c 
ime an under s 
Actively in t 


ing ws t omplish these things the approachto teach- 
Cipline radically changed. No longer was the dis- 
fashio, presented in lecture and in text in didactic 
Place where longer did the laboratory serve asa 
ere the student demonstrated to himself 
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facts already known. 


Instead, the laboratory has be- 
come the place where 


facts are learned through in- 
vestigation and the classroom the place where the 


results of laboratory investigations are discussed 


and an understanding of fundamental Scientific con- 
cepts are developed. 


The Physical Science Stud 
led off the revolution in 1956, This Broup decided 


In a similar w 


ay, the other high- 
lum groups point 


A School curricu- 
ed their courses in ч 


new directions. 


(CBA) 
ientists and hi 


and that modern chemical concepts i 

tioned and little used. The СБ e rig aa men- 
make the most fundamental of all chemical che 
the central theme o; Concepts 
of chemical bonds has 

the curriculum. Chemi 
play of experiment and ideas – о 


as 
fw i 
and what is thought hat is 


- Inthe laboratory Progr 
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students carry out small investigations, and in the 
classroom they learn to apply chemical concepts to 
explain their observations. 


The Chemical Education Materials Study (CHEM 
Study) also stresses observation and concepts. This 
study grew out of the recommendations of an ad hoc 
committee set up by the American Chemical Soc iety 
to consider needed changes in high- school chemis- 
try. The philosophy of the CHEM Study group is 
that the important concepts and generalizations of 
chemistry should be developed inductively and that 
the evidence that the students use should be gathered 
by them in the laboratory wherever possible. Where 
this is not possible it is presented by teacher dem- 
onstrations or films. There is emphasis ontheuni- 
fying concepts in chemistry rather than on great vol- 
umes and varieties of technical applications. 


The Biological Sciences Curriculum Study (BSCS) 
of the American Institute of Biological Sciences 
faced some problems that were more touc hy than 
those faced by the physicists and chemists. Tradi- 
tionally, the topics of evolution and of human repro- 
duction had been taboo in high- school bio logy 
courses. The BSCS group felt that modern biology 
could not be taught without introducing these topics, 
and they, therefore, included themin the three 
courses which they developed. Though there has 
been some resistance to these controversial topics 
in a few places, there is now general acceptance of 
the new courses by schools throughout the country. 
Indeed, it is estimated that this year 80 percent of 
all students taking biology in the tenth grade are in 
one of the BSCS courses. Though the three BSCS 
programs take different approaches to the teaching 
of biology, the goals are the same - to present bi- 
ology as a dynamic and growing field of knowledge, 
not as an encyclopedic catalog of facts. A major 
accomplishment of the BSCS grouphas been to 
change the high-school biology laboratory from а 
place where dead, preserved s pecimens are dis- 
Sected to a place where living organisms are exam- 
ined and experimented with. 


In every case the curriculum groups saw to it 


that the materials wer 
е thoroughh i x 
rooms before being re yp с 


School students could, indeed, 


in th 
been gratifying. ese new ways. The results have 


К е important accomplishments of all of 
um groups has been tobring together 
nd teachers to work out the n ем pro- 

are notcourses handed to the 
ege professors, but aretruly pro- 
€ been developed cooperatively, 


nis developing anew 
* ы is hoped, willencour- 
€ physics, The pro- 


ject Physies, adopts a 


cultural approach. It develops important concepts 

such as force, energy, and motion, but draws heav- 
ily on the humanistic background of the science. 
The program is still in an experimental stage, and 
it remains to be seen whether it will achieve its 


goal of increasing physics enrollment in high 
schools. 
The immediate short-range goals of the com- 


pleted high-school curriculum projects seem to have 
been reached in a satisfactory manner. Through 
tryout and evaluation it is clear that high-school stu- 
dents can learn science through the medium of the 

new programs. There are some areas, however, 
where further research remains to be done. Each 

of the curriculum groups either explicitly or implic- 
itly suggest that their approaches should develop in 
students an understanding of and an ability to engage 
in the scientific enterprise, and an interest inand 
enthusiasm for science which should carry on be- 
yond the high-school years. Little has been doneto 
determine whether the programs do indeed have this 
effect. A long-range study to evaluate this aspect 
of the new curriculum programs would be desirable. 


Another area which should be investigated is the 
effect of one program on another in a school. For 
example, one might ask whether a student in PSSC 
physics in the twelfth grade does better if he has 
previously had BSCS biology and CBA or CHEM 
Study chemistry than if he has not. The AAAS Com- 
mission on Science Education has plans for carrying 
out such a study but the plans have not been imple- 
mented, 


A third and important question is one of sequence 
of the courses in high school. The curriculum 
groups designed their programs to fit into existing 
slots: biology, tenth grade; chemistry, eleventh 
grade; and physics, twelfth grade. This sequence 
of courses has become established over several 
decades but there is no logical reason for teaching 
the courses in this order. Indeed, when one looks 
at the interrelationships of the disciplines, logic 
suggests that the order should be inverted, Modern 
chemistry is deeply rooted in physics, and modern 
biology requires of the student more thana speaking 
acquaintance with chemistry. The groups who pre- 
pared the chemistry and biology curricula rec og- 
nized the problem. The chemists were forced to in- 
clude some of the ideas of physics -for exa mple. 
the concept of energy – in their programs. The bi- 
ologists in turn, had to include some simple chem- 
istry in the hope that students would better under- 
Stand some of the molecular biology which is in- 
cluded in the courses. The result in each case is 
not completely Satisfactory, and it seemsclear that 
Some research needs to be done on this problem. I 
Shall deal with this later. But now I want to turn to 


the pre-high school years and discuss the develop- 
ment of science curricula there. 


PRE-HIGH SCHOOL 


I said earlier that the new high-school science 
programs have set the 
lum research in the pre- 
tern was set in several 
of scientists and teache. 
grams, and emphasis o 
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and investigation rather than on didactic presenta- 
tions. As the new high-school curricula came into 
being, the writers and the teachers who used them 
soon realized that many of the skills and attitudes 
that students need for effective participation in the 
new curricula should be learned in earlier grades. 
Let me take just one example. The sc ientists work- 
ing with high-school teachers developing CBA chem- 
istry soon learned from the teachers that they could 
not assume that students entering high- school chem- 
istry had developed skills in constructing and inter- 
preting graphs and so a section of the CBA program 
is devoted to developing these abil ities. There 
seems to be little doubt that these are skills that 
could be developed in earlier grades. 


Many of the scientists who worked on the high- 
school curricula are now devoting their energies, 
in collaboration with school teachers, to designing 
and testing science curricula for the early grades. 
There are so many of these programs for ele- 
mentary and junior high school that we cannot exam- 
ine them all in detail. But we will review most of 
the major ones Starting with the programs for the 
early grades and telling you in some de tail about 
the elementary science Program on which I am 
working. 


Each of the groups working at the eleme ntary 
School level have the same general philosophy and 
plan of attack, The purpose of introducing new sci- 
ence programs into the early grades is not so much 
to recruit more scientists as to prepare individuals 
to be Scientifically literate citizens. Literate not 
in the sense of knowing many scientific facts, but 
of understanding both the power and the limitations 
of science and its fundamental concepts. 


The approaches that the various curriculum de- 
velopment groups are taking to achie ve this goal 
vary widely. The Elementary Science Study of Ed- 
ucational Services, Incorporated takes the position 
that the child should be introduced to sci 
senting him with a variety of materials 
gate. They have developed and te sted inclass- 
rooms, units on butterflies, gases and airs, kitchen 
physics, microgardening, meal worms, and many 


other topics. Let us look at the meal worm unit as 
an example, 


The children are given shoe 
few meal worms and are left to 
their own, Gradually, 
begin to ask questions. 
find the edges of the box? How do they find their 
food? How do they back Up?... and so on. As they 
design experiments to answer their Questions, it is 
hoped that they will learn to evaluate evidence based 
on Observation, Classification, measurement, and 
Contro] of variables; that they wi 
tions that they can answer by simple tests; and that. 

the same time, they will develop an enthusiasm 


The School Science Curriculum Project of the 
a ersity or Mi 
ESS. 


that of nois has a philosophy similar to 


The Sci 
the y Science 


Curriculum Improvement St udy at 
California takes a different ap- 


Niver sity of 


proach. Its main thrust isto develop an understand- 
ing of science concepts starting inkindergarten with 
the simple concept of **What is an objec and mov- 
ing from there to “What isa system?", **What is 

meant by interaction?", and so on. Their program 

is more structured than ESS but still leaves room 

for investigation by children, 


A science program for the early gradesis the 
Minnesota School Mathematics and Science group, 
abbreviated **Minnemast." This group began by de- 
veloping a mathematics program and then devising 
Science units to go along with it. Theirscience pro- 
gram aims at developing both skills and concepts. 
So far they have seven units for kindergarten and 
first grade entitled "Watching and Wonder ing, 
“Describing and Classifying, “ “Our Senses, `' 
‘‘Shape and Symmetry," “Objects and Their Prop- 
erties, ’’ “Changing and Unchanging Pro perties," 
and ‘‘Introduction of Measurement. `` 


The project with which I am working is develop- 
ing a program, called Science - A Process Ap- 
proach, for K through grade 6. This curriculum 
development is one of the activities of the AAAS 
Commission on Science Education. Shortly after it 
was established in the Spring of 1962, the AAAS 
Commission on Science Education held two confer- 
ences of scientists, science educators, and teach- 
ers. From these conferences came the suggestion 
that a fruitful approach to teaching science in the 
early grades would be to devise exercises which 
would develop children's skills in using the proces- 
ses of science. The Commission elected to carry 
out this task. This program, which is now be ing 
tried out in schools for the third year, is, like the 


other projects, a cooperative effort of scientists 
and school teachers, 


A basic premise 


tist’s behaviors as 
tute a complex set of skills 


er prem- 
to use these vari- 


ous processes can be developed ina Step-wise fash- 


3 recognizing and usin 
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as more than, fewer than, the same number 

as, most, and fewest. 
There is a section called Rationale which tells the 
teacher why graphing is introduced at this level. It 
gives her some suggestions about contructing bar 
graphs and also includes suggestions of how to han- 
dle children's questions about why a graphisuseful 
in communicating information. 


The activities are introduced by putting into a 
box a number of colored blocks so that there will be 
one for each child and so that there will be at least 
three different colors. The teacher then asks each 
child to come and pick a block out of the box and then 
put it into a rough pile on the table sothatthe blocks 
of each color will be piled together. The teacher 
then asks the children if it is easy to find how many 
blocks there are of each color. The children, of 
course, find it necessary to come upto the table 
and count the blocks in the piles. The teacher then 
asks if they can think of a way of arranging the 
blocks so that the number can be found more easily 
and suggests, if the children do not think of it, that 
if the blocks are piled in vertical columns, they can 
be counted easily even from a distance. This leads 
ultimately to the question of ways of remembering 
how many blocks there are of each color and into 
the next activity in which bar graphsare first intro- 
duced. In that activity each child again selects а 
block and also a felt square and as he putsthe block 
in one of the piles on the table he puts the felt square 
in one of the columns on a felt board. At the end of 
this activity there is a bar graph consisting of felt 
Squares on the felt board, one column for each col- 
or of block. In the activities that follow the chil- 
dren make bar graphs by coloring squares on large 
sheets of paper and finally are introduced tothe idea 


of a grid and of labeling the vertical axis of the grid 
with numerals and the horizontal axis with a key to 
identify the objects, 


At the end of th 
the teacher uses t 
have developed th 
For the Арргаіѕа! 
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terpreting data, devising models, and making oper- 
ational definitions. We sometimes speak of these 
as the integrated processes because they bring to- 
gether the skills from the earlier part of the pr o- 
gram. 


One of the advantages of a program in which the 
objectives are stated in behavioral terms is that 
evaluation can be done by assigning the children 
tasks which will indicate whether or not they have 
attained the expected skills. For example, follow- 
ing the kindergarten exercise in which the children 
learn to identify and to name various plane shapes, 
such as circles, squares, triangles, and rectangles, 
it is a simple matter for the teacherto present a 
child with an array of plane shapes and ask him to 
point to a triangle, or the teacher might point to a 
circle and ask the child to name it. The results of 
each task can be recorded with a binary scale. Yes, 


the child could perform the task, or no, he could not. 


For each exercise we have prepared a series of 
tasks which we call the competency measure. Each 
teacher in our tryout program uses the competency 
measures on a sample of children from her class 
after she has taught each of the exercises. The re- 
sults of this testing in the various tryout centers 
are collected and tabulated and used by the writers 
as they revise the exercises during each summer 
writing session. The results give the writers a 


good idea about the success or failure of any given 
exercise. 


There are many aspects of the teaching in the 
classroom that we cannot learn about through the 
competency measure and so we ask each teacher to 
give her subjective impression of the exercise on a 
feedback form. This includes information about the 
time required to carry out the instruction, sugges- 
tions of activities that might be omitted or added, 
the reaction of the class, including interest and en- 
thusiasm, carry over of learning to other activities 
or areas, and attention span. This feedback is also 


helpful to the writing group as the program is re- 
vised. 


Although the emphasis in Science - A Process 
Approach is on developing skills in using science 
processes it is important to ask the question, ‘‘What 
about science content?" It is obviously impossible 
to teach process skills without using content of 
Some sort, and the content that we use is drawn 
from various fields of science. To insure this we 
include in our writing team scientists from all ma- 
jor areas. In the early grades, of course, there 
are many exercises in which the relationship of the 
content to science appears to be somewhat distant. 
Learning to recognize and name tw o- dimensional 
and three-dimensional shapes isanexample. Learn- 
ion oo using colored blocks is another. 


€ program pro, i igher 
grades the content is Dou aaee into mgn 


are the following: 
Inferring 8 — Inference i i 
Bia S of Patterns in Electric 
Defining Operationally 1- 
and Nonconductors 


Controlli i 
BLUR Variables 1 – Growth of Mold on 


Circuits, Conductors, 
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Interpreting Data 1 - Guinea Pig Learning in a 
Maze 

Controlling Variables 7 — Orientation of Plants 

Experimenting 1 — Separating Mixtures 
Of the eighty-one exercises in Parts Five, Six, and 
Seven, the subject matter is approximately as fol- 
lows: physics 33 percent; biology 21 percent; math- 
ematics 17 percent; chemistry 9 percent; psychol- 
ogy and social science 9 percent; earth science 7 
percent; astronomy 4 percent. 


JUNIOR HIGH SCHOOL 


We believe, then, that by the end of the sixth 
grade, children going through this program will not 
only have developed sufficient skills to carry out 
simple investigations but also will have a store of 
scientific knowledge which will make it possible for 
them to embark on a fairly sophisticated science 
program in the junior-high- school years. What the 
pattern for junior-high-school science will turn out 
to be is not yet clear, At the moment there are 
four groups who have developed curriculum materi- 
als for junior high school and tested them ona fairly 
wide scale. Then there are several other groups 
who are at earlier stages of curriculum develo p- 
ment. The four programs that are farthest along 


are centered mainly in the physical and earth sci- 
ences and astronomy. 


The astronomy program, knownasthe University 
of Illinois Elementary School Science Project, ac- 
tually is designed for the later elementary grades 
and the early junior-high- school grades. This pro- 
gram is frankly subject matter oriented and con- 
sists of a series of childrens’ books and teachers’ 
guides. The program starts with the topic **Chart- 
' which includes measuring dis- 

System, measuring the size and 

» and using light asatoolfor meas- 
uring great distances, The topics which follow it 
аге “Тһе Universe in Motion, " « Gravitation, ” and 
“Тһе Message of Starlight. ” Two additional topics 
which are still in preparation аге “Тһе Life Story 
of a Star” and “Galaxies and the Universe. " These 
materials are not intended to serve as a complete 
science program for grades five through eight, but 
rather as topics to be included in a broader science 
program. The materials are being tested by sev- 
eral hundred teachers in various parts of the coun- 


try but the results of the evaluation have not been 
reported, 


ninth grade, This course was first wr 


team of scientists and teachers in the summer of 


eing prepared for commercial pub 


; and energy flow in 


paleobiology. The course ends w ith a fourth unit, 
“Earth and the Universe, " which begins with a dis- 
cussion of the moon, moves on to the solar system 
and its origin, and finally to stars, galaxies, and 
cosmology. The program has been ev aluated in 
eighty classrooms, and student progress has been 
determined with a test on science knowledge which 
is used both as a pre- and posttest. 


À third junior- high- school program - Time, 
Space, and Matter - is being developed by the Sec- 
ondary School Science Project at Princeton Univer- 
sity. This program begins and ends in geology but 
includes a good deal of fundamental phy sical science 
as well. The students investigate topics such as 
solution, crystallization, and water erosion using 


well-designed equpiment which is furnished to them 
in the form of individual kits, 


A good deal of attention is being given to eval- 
uating this program to provide feedback for those 
who are revising the course. Each teacher keeps a 
daily record of course progress and reports at reg- 
ular intervals to the project office. The teachers 
are particularly requested to indicate where stu- 
dents show an extension of knowledge or interest to 
out-of-class situations and where they show signs 
of understanding or of confusion as they use the 
course materials. In addition, the groupis develop- 
ing a test to demonstrate changes in the behavior of 
children after they have had experience with the new 
Curriculum. Other methods of evaluation, including 


the instructional interview technique, are being con- 
sidered. 


we are a long way from a Smooth flo 
curriculum for these grades. There 
or no coordinatio: 


TOgBress gj 
hem, 
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At the conference, Bentley Glass and Arnold 
Grobman of BSCS suggested two possible t y pe s of 
junior-high-school science programs. One is a top- 
ical plan and the other a stratified plan. In the top- 
ical plan they suggested that the science program 
for grade 7 include about half a dozen topics, such 
as conservation of matter and energy, natural cy- 
cles, biological regulation, and homeostasis. An- 
other half dozen topics would be developed for eighth 
grade. These would include change through time - 
cosmological, geological, and biological. Another 
group of topics would be developed for ninth grade 
and would include the broad aspects of meteorology, 
environment, and habitat. 


The stratified plan that they suggested was as 
follows: 

Grade 7 – physical science - a year course in 
descriptive aspects, largely earth sciences. 

Grade 8 - life science — a year course in diver- 
sity and unity in the organic world, fundamen- 
tal structure and function. 

Grade 9 – physical science - а year course in 
basic concepts of matter and energy. 


Another possible approach to an integrated jun- 
ior-high- school science program was described at 
the conference by Dr. Robert Jastrow, chairman of 
the Junior High School Science Subcommittee of the 
AAAS Commission on Science Education. This pro- 
gram would follow naturally from the elementary 
program, Science – A Process Approach, but could 
also build on the other elementary science programs 
that I have already described. The course for sev- 
enth grade would serve as a transition between an 
elementary science program where the student has 
developed skills in using science processes and 
where he has learned some basic science content; 
and the eighth and ninth grades in which the course 
would develop a coherent picture of man and his 
physical and biological environment. The seventh- 
grade course, which would be rooted in laboratory 
experiences and field trips, would consist of selec- 
ted topics from the physical and biological sciences 
and from astronomy and the earth Sciences. 


The eighth grade would begin with consideration 
of the basic materials and forces of the universe 
The neutron, proton, | 
units out of which 
basic forces of nai 


of the birth of star 
of chemical elements. 


; and finally the course would take 
5 of molecular biolo 


oundations of evolution, 


The ninth- 
the point of таче cou: 


rse would treat biology from 
Considered y, 


iew of evolution The fir i 
E St topic to 
Оша be the development of Beli 2-04 


Н 


cating cells from primitive catalysts and complex 
organic chemical systems. Then multicellular or- 
ganisms would be discussed and the similarities be- 
tween plant and animal forms as far as capture, stor- 
age, and step-wise release of energy is concerned. 
Following this, differentiation and embryogenesis 
would be considered and then the molecular, organ- 
ismic, and ecological factors involved in natural 
selection would be treated. The ninth-grade pro- 
gram would end with a discussion of the development 
of the nervous system and with how the nervous sys- 
tem accounts for consciousness, perception, learn- 
ing, and memory. 


A course of this sort, in which central problems 
of modern science serve as the core, should serve 
as a general education course for students who do 
not elect to take much science in high school. At 
the same time, it should provide a solid background 
of the skills and understanding which would serve 
as an effective base for the high-school biology. 
chemistry, and physics courses. No work beyond 
planning has been done on this program yet. It is 
hoped that work can begin within the next year. 


SCIENCE COURSE SEQUENCE 


Now I want to come back tothe question about the 
sequence of science courses in high school that I 
have already mentioned briefly. One of the argu- 
ments for putting physics off until the twelfth grade 
is that the students do not develop the necessary 
mathematical skills until they have reached that lev- 
el. I have not said anything about the new math- 
ematics curricula except the relationship of mathe- 
matics to the Science - A Process Approach ele- 
mentary curriculum. There are, of course,sev- 
eral new approaches to teaching mathematics in the 
Schools. In these new programs the students devel- 
op mathematical skills at muchearlier grade levels 
than they have done in the older traditional pro- 
grams. It is possible, I think, that by the end of 
the ninth grade, students will have sufficient skill 
in algebra to do the mathematics in a modern phys- 
ics course such as PSSC. Mathematics skill would, 


therefore, no longer be a blockto moving physics to 
an earlier grade. 


If а junior-high- school science curriculum gives 
the student some fundamental knowledge of physics. 
chemistry, and biology and related sciences, I see 
no reason why physics cannot be made a tenth- grade 
course, followed by chemistry in grade 11, and а 
molecular biology course in grade 12. Already, i” 


a number of school systems around the country tie 
Sequence is being tried. 


There is another possible pattern for grades 10 
and 11. Since there are a number of topics that are 
common to physics and chemistry, it is possible {0 
interweave these two disciplines into a two-year 567 
quence of physical science. An experiment is now 
under way in Portland, Oregon, to do just this. A 
group there set out three years ago to devise a “гоа 
map” through PSSC physics and CBA chemistry and 
a similar one through PSSC physics and CHEM 
Study chemistry so that teachers could move easily 
from one book to the other. The program starts 
with the topic “Interpreting the Universe, '' and con^ 
Siders fundamental Concepts and measurements and 
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the nature of chemical change. The next section is 


mechanics which is all drawn from the PSSC course. 


The rest of the year is devo 
ture of matter, atomic models, and molecules and 

energy. In all of these chemistry and physics are 

interwoven. In the second year of the program there 
is less blending of the two Subjects. The course be- 
gins with optics and waves, followed by electromag- 
netics and a discussion of quantum systemsand then 
moves into consideration of bonds and chemical sys- 


tems; order, disorder, and change; and molecules 
and energy, 


ted to the electrical na- 


This program has been evaluated using selected 
test items from CBA, CHEM Study, and PSSC tests. 
The Portland group reports that the combined pro- 
gram is successful and that even though students 
are permitted to take only the first year if they like, 
a large majority of them Stay with it for the fulltwo 
years. At present, this program isdesigned for 
grades 11 and 12, but it could be used at grades 10 
and 11. 


The ‘‘road map’’ technique is not completely sat- 
isfactory, and there is need for a fresh start to de- 
р an integrated modern physical science curric- 
ulum. 


REVIEW OF PROBLEMS 


We have examined some of the recent and current 

3 curriculum research and have Suggested di- 
Science in which curriculum research might go in 
rections 1 There are some corollary problems 
елаша omitted. We will review these briefly. 

a 


а science education то ves from 
(As nde e through books and exposition 
fairly pos antpulatioh and investigation, there will 
to studen easing need to provide supplies and equip- 
ee or classrooms even as early as kindergarten, 
ment for са f money provided for science supplies 
The ере systems is small. It will have to be 
momona d manyfold to accommodate the new pro- 
inc FORET impossible to do more than estimate 
grams. r Pike cost of introducing the new elemen- 
roughly wh sia wouldbé. ‘Ik appears that oq the or- 
tary progr r classroom will be needed to intro- 
der of $200 D Ебра. To assist school systems 
duce the nee the introduction of the new elementary 
in Planning rams, the AAAS Commission on Science 
E nia P is dusk embarking on a study of the costs 
"ded and supplies. It is our ex pectation 
"iaa study will be completed shortly. 


Classroom design will also be influenced by the 
new science curricula. As a minimum for the early 
grades, classrooms should have multiple electrical 
outlets, running water, and level working areas. I 
have been surprised at the many classrooms I have 
visited that have only a single electrical outlet. 


Although the new Junior-high-school science cur- 
ricula that are now under development are designed 
to be used in classrooms without special laboratory 
facilities, I think that in the future junior -high- 
School classrooms will need to be provided with 


laboratory space similar tothat found in senior high 
schools. 


А crucial problem raised by the new curricula is 
teacher training. As you know, the sen ior-hi gh- 
School science teachers have been able to attend 
summer institutes to learn to us 
chemistry, and physics courses. 
are now being held for junior-high- 
teachers. For the elementary schools the problem 
is different both quantitatively and qualitatively. The 
arliest grades do not re- 


School science 


re widely used, school 
ice training programs. 
Science teachers 
grams. 

And then there is the problem of preservice 
training of teachers. Only one of the curriculum 
groups, *'Minnemast, `“ is preparing college mate- 
rial to train prospective teachers to use the **Minn- 
emast” materials. There is а group at Rensselaer 
Polytechnic Institute that is designing a new course 
in physical science for non-science majors with the 
thought that this course will better prepare new 
teachers to use the new school science curricula, 


needed and Significant change 
Changes which could not have 
Scientists alone or 
tion must, 


been effected by either 


This coo pera- 
» Continue, 


=== Curriculum Research and 
Development in English 


THIS PAPER is planned to present 
selected items of recent researchinthe teaching of 
English from a number of aspects. Together they 
make up the body of information and expert opinion 
Which will largely direct curriculum work in the 
decades ahead. Although the goal of most English 
teaching is to accomplish an integration of the com- 
Ponent parts of the Subject, they are here presented 
in divisions to avoid confusion and o verla p. One 
area of research and study often associated with the 
English language arts is reading as a skill. It is 


omitted from the surve: because in i a 
cation it is Considered A ke id 
arts. Ithas its own co! 


i mselves research 
чсе research in the are 


THE PROFESSION 
OF ENGLISH TEACHING 


dations for th 


g 
lish by meeting 2070уешепі of th 


ing recognized needs, 
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e teaching of Eng- 


Robert C. Pooley 


The University of Wisconsin 
Madison, Wisconsin 


The scope of the responsibility of Englishisdram- 
atized by a bar chart revealing that of all high 
School students in school in 1960, 92. 9 percent were 
enrolled in English courses, 68 percent in social 
Sciences, 55 percent in mathematics, and 25 percent 
inforeign languages. A survey revealed that in the 
years 1958-1959 the demand for trained high- school 
teachers outran the supply by 27 percent. Ex pand- 
ing school populations will increase this ratio. 
Moreover, the expansion of knowledge and the re- 
quirements of a complex Society will increasingly 
demand more technicaltraining and knowledge on the 
part of English teachers than was true in the past. 
The single greatest weakness in the current situa- 
tion is the lack of articulation in the teaching of Eng- 
lish from the elementary school through college. 


A summary of the highlights of the status report 
follows: 
The Need for Teachers of English: | 

a. The shortage of teachers shows no sign of 
decreasing. 

b. One-fourth of all elementary teachers are 
not college graduates. 

c. Only 40 to 60 percent of high-school Eng- 
lish teachers have completed a college ma- 
jor in English. 

d. There is an acute shortage of college in- 
Structors in linguistics and composition. 

State Certification Laws: 

a. States differ widely in requirements. 

b. Ten states certify elementary teachers 
without a degree. 

€. Nineteen states have no stated requirement 
in English for elementary teachers; twenty- 
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one lack definite requirements in reading, 
children’s literature, and English methods. 

d. Minimum preparation in English for Eng- 
lish teachers is sixteen to eighteen credit 
hours. 

Preparation of elementary teachers: | 

à. More {һап 94 percent of the colleges fail 
to require specific study ofthe English 
language. 

b. Less than half require a course inthe teach- 
ing of the language arts. 

Preparation of secondary-school teachers of Eng- 
lish: 

à. Only one-fourth of the colleges require a 
course in the history of the English lang- 
uage. 

b. Only 17 percent require a course in Modern 
English Grammar. 

c. Only 41 percent require a course inad- 
vanced composition. 

d. Only slightly more than half the colleges 
require a course in the methods of teaching 
English. 

Better Teaching Conditions: 

Teachers with a student load of 150 (five sec- 
tions of thirty students) must work more 
than 50 hours a week just meeting classes 
and marking papers. (In 529 out of 682 
Schools, English teacher shad five classes. ) 

Libraries: 

a. 10,000, 000 elementary, and about 150, 000 
high- school students attend Schools lacking 
à central library. 

b. The average expenditure for books per stu- 


dent is about half the price of one average 
book. 
Remedial English in Colleges: 

a. An estimated 150, 000 students failed college 
English tests in 1960. 

b. Two-thirds of American colleges have to 
offer remedial work in English. 

€. The cost of remedial instruction is esti- 
mated to be over ten million dollars. 


A continuation of the research into the status of 
the teaching of English was carried on by acommit- 
tee of the National Council of Teachers of English 


y 7,417 Secondary- school English 
chool teachers. 


Some highlights of the analysis are: 

1. Over 40 percent of the nation's elementary 
teachers began full-time teaching without a 
degree, Only 31 percent of these achieved the 
degree after beginning teaching. 

2. More than one in ten high-school teachers 

11.8 Percent) began full-time teaching with- 
out a degree, Practically all of these earned 
a degree later. 
3. Although 24 Percent of instructional time in 


given to this subject, 


4. About half (49.5 percent) of secondary teach- 
ers teaching English lack a major in this sub- 
ject. 

5. By their own admission only 51.9 percent of 
secondary-school English teachers feel com- 
petent in literature; 36. 6 percent feel compe- 
tent in composition, and 53. 5 percent feel com- 
petent to teach the English language. 

6. In voluntary continuing education, elementary 
teachers take one summer session for each 
eight years of experience. But the courses 
taken are largely in education, with little or 
no further education in the language arts. 

Л. Thirty percent of secondary teachers of En g- 
lish have not taken a course in English in more 
than ten years, 

8. Interest in further education for elementary 
teachers was expressed for Six areas: chil- 
dren's literature, language, Speech, writing, 
general literature, and the largest number, 
79. 1 percent in reading. 

9. Secondary teachers expressed interest in fur- 
ther experience in practical methods of teach- 
ing English: intermediate or advanced compo- 
Sition, literature for adolescents, teaching of 
reading, literary criticism, and structural 
and/or generative grammar. 


The broad conclusion is overwhelming evidence 
of inadequate training with a clear need for means 
to make inservice further education available to 
nearly all teachers of the Elish language arts, 


Two books of importance to the whole profession 
of the teaching on English appeared in 1963 and 1965 
respectively. The Education of Teachers of English 
(Grommen, 1963) is Volume V of a curriculum se- 
ries of the National Council of Te 
Written by a committee represen 
school, high-school, and college 
it analyzes facts, implications, a 


(1965). The Commi 


ers in all parts of the United States rs reach 
Sioned and issued à Series Of kinesco meee 
perienced college teachers рза 
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charge except for mailing costs. The volume Free- 
dom and Discipline in English is the final report of 
the Commission. 


Chief contributions to the curriculum are: 

1. Recommendations to improve the quality of 
instruction. 

2. An analysis of language teaching, including 
grammar and usage. 

3. An analysis of literature teaching, emphasiz- 
ing the critical process in teaching and study- 
ing literature. 

4. An analysis of composition, including valua- 
ble suggestions for motivating writing, and for 
the correction, Criticism, and measurement 
of writing skill. 


Without essential conflict in fundamental matters, 
it may be stated that, in general, the publications 
of the National Council of Teachers of English tend 
to be experimental and progressive in outlook, and 
that the point of view of the Commission on English 
is conservative. To the curriculum director there 
is great value in the balance to be derived from 
these differing viewpoints. 


RESEARCH IN THE 
TEACHING OF LITERATURE 


Although most of the studies in literature deal 
with ideas and values rather than with controlled 
evaluation, some recent studies reveal effective 
techniques for measuring the success of students in 
the learning of kinds of literary materials. Squire 
(1963) studied the responses of adolescent readers 
to four short stories, establishing their responses 
as forming seven categories: L Liter ary Judg- 
ments: Direct or implied judgments on the story as 
an artistic work; П. Interpretational Res ponses: 
Reactions in which the reader generalizes and at- 
tempts to discover meaning, motivation, and the na- 
ture of the characters; III. Narrational Responses: 
Reporting facts or details from the story without in- 
terpretation; IV. Associational Responses: Re- 
Sponses in which the reader assoc iates ideas, 
VC eee or people with his own experience; 
HR associata ы Responses in which the read- 
Женка. sd with the behavior and/or emo- 
Respohées’in HER IS VI. Prescriptive Judgments: 
Si action basen zs the reader prescribes a course 

Some absolute standard; VII. Mis- 


cellaneous Responses: all others. 
Wilson 


(1966) reports the responses of college 


freshmen to the readin 


actions in class, 
they wrote follow- 
followed tor each 


esponses were с 
categories, 
65. 6 percent of 


Oded by means of 
Statistical ana lysis 
responses to the three 


The lead of interpretation over literary judgment is 
a sign of maturity of perception; it suggests that 
teachers should avoid supplying *'correct'"' interpre- 
tations before the book is read by the student. The 
study reveals that although students may begin their 
involvement with literature in a groping and emot ion- 
al fashion, their later attempts at interpretation be- 
come sharply formulated and logical. Teachers 
should therefore refrain from demanding that first 
reactions be fully described or defended. But as the 
study of the novel moves into the grappling with dif 
ficulties, the instructor should demand expert anal- 
ysis. Most students are capable of growth of inter- 
pretation by these means. 


The Research Foundation of the National Council 
of Teachers of English is sponsoring, directing, and 
supporting research in children’s growthin sensitiv- 
ity to, and appreciation of literature. Starting with 
a national conference of elementary- school, high- 
School, and college specialists in literature assem- 
bled in Chicago in May 1963, the Foundation applied 
itself to research in the teaching of literature be- 
cause so little is being done. It was soondiscovered 
that there were no instruments available to measure 
any aspect of success in children's growth in litera- 
ture. The Foundation then determined to try to cre- 
ate such tests, both as research in itself, and as the 
means to further studies. Following consultations 
with various authorities, the Foundation appointed 
Margaret Early of Syracuse University the Director 
of the test activity and appropriated funds in its sup- 
port. In the summer of 1965, Early assembled at 
Champaign, Illinois, some twenty-five selected 
teachers and librarians to develop test items. From 
the materials produced by this group, and augmented 
by Early's advisors, four forms of a test have been 
developed by cooperative work with the Educational 
Testing Service of Princeton, New Jersey. A trial 
run was made of these formsinthe spring of 1967 at 
elementary schools in grade 5 in locations covering 
the United States. From the statistical study of the 
returns it is expected that two final forms of the test 
will be developed. 


The goal is to attempt to discover growth in the 
responses of children to literary experiences. The 
range is from grade 4 to grade 6, inclusive. Al 
though it will be a pen and paper test, many of the 
items will be presented orally on tapes. If the test 
reveals measurable growth in responses to lite ra- 
ture, the research will continue in two directions: 
(1) to measure the effectiveness d kinds ET 
tion on the literary growth of children; a 4 
create similar Da junior-high schools and pos 
sibly Senior-high schools. 


RESEARCH IN THE 
TEACHING OF COMPOSITION 


One of the most exhaustive researches of donde 
times was conducted by Richard Braddock, Richard 
Lloyd Jones, and Lowell Schoer of the University 
of Iowa and published in 1963 by the NCTE under the 
title Research in Written Composition. It undertook 
to assess all known research in c omposition that 
could be described as ‘‘scientific. " From twenty 
bibliographies of research came over a thousand ci- 
tations; committee analysis reduced these to 485 
items. Fromconsultation with seventeen authorities 
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this number was reduced to about one hundred titles; 
later some fifty other titles were added. After vig- 
orous analysis, only five studies survived the crite- 
ria set up by the investigators. These five only, in 
the eyes of the investigators, can be termed valid 
in the teaching of composition. 


1. The Baxton study revealed that thorough crit- 
icism of composition by a competent reader, fol- 
lowed by revision by the student brought better gains 
than writing not criticized. 


2. The Harris study showed in a 2-year space 
that the direct (or functional) teaching of grammar 
brought better results than the formal, analytical 
method. 


3. The Kincaid study undertook to reveal whether 
à single paper written by a student on a given topic 
at a particular time provided a valid means to judge 
a student's writing ability. Joined to this question 
were several others, so that clear answers were 


almost impossible to secure because of weakness in 
the structure of the research. 


4. The Smith study dealt with class size asafac- 
tor in the teaching of writing. It revealed that a 
large class of superior ninth-grade students can 
progress in composition as well as a small class. 


5. The Becker study compared three methods of 
teaching, described as normal, bibliographic, and 
kinescope methods, The so-called normal method 


was found to have a slight superior ity over the 
others, 


The conclusion of the Braddock investigations is 
that of the more than a thousand items originally 
noted, only these five may be considered to yield 
valid answers from sound research. 


In a review of the Braddock report inCollege 
English, Jean Hagstrom (1964) analyzes the re- 
search methods employed in these five researches, 
and concludes, “И is therefore extremely disheart- 
ening to say that 1) none of them strikes a layman 
as definitive or persuasive, and 2) there is very lit- 
tle promise that without rigorous antecedent thought 
the ‘scientific’ method applied to composition will 
yield better results in the future than it has in the 
past." He urges antecedent thought on such basic 
problems as: How do you find a subject? How do 


you clothe it in appropriate style? How do you give 
it effective organization? 


The Research Foundation of the National Council 
of Teachers of English planned a 1967 conference of 
Specialists in a number of fields to attempt to iden- 
tify the fundamentals of composition. 


Current resear 


ch in composition has taken in- 
terestin, р » 


Б and profitable direction in applying statis- 
9cedures to rhetoric, Kellogg W. Hunt 
1965b) collected large samples of students’ 
ted to du des 4, 8, and 12. These were subjec- 
differe ural analysis to attempt to describe the 


if : 
Ces in sentence Structure as students mature. 
001 of ana] 


the «т-ц? Sls Hunt developed what he called 
each ter nit”: а group of words minimal in length, 
minated grammatically between a capital 


letter and a period. This unit has the advantage of 

preserving all of the subordination achieved by a stu- 
dent, and all of his coordination between words and 
phrases and subordinate clauses. 


His findings, very much condensed, are shown 
in Table 1. This table reveals that the most signif- 
icant mark of maturity in writing is not sentence 
length in itself, but the increase of length of the T- 
units. When superior adults writing for Harpers 
and the Atlantic Monthly were analyzed by the same 
means, the length of their T- units increased by 40 
percent over the twelfth graders’, anincrease equal 
to the gain of twelfth graders over fourth graders, 
Twelfth-grade students came close to the profes- 
sionals in the number of clauses they use; the 
growth among professionals is inthe increased num- 
ber of words per clause. These results giveaclear 
indication of the factors to be studied and taught in 
the improvement of sentence structure, 


Francis Christensen (1963b) counted sen tence 
Openers to provide objective knowledge of how pro- 
fessionals begin sentences. High- school and college 
composition teachers have generally encouraged a 
wide variety of sentence openers, including prepo- 
sitional phrases of all types, participial phrases, 
gerund phrases, absolute constructions, and sen- 
tence inversions. One writer says, “When a stu- 
dent opens a sentence with an infinitive phrase or a 
past participle, we immediately stamp him as more 
mature...” 


Christensen analyzed twenty recentor contempo- 
rary writers - ten of narrative and ten of expository 
writing — to determine by what constructions they 
open sentences. The findings are in direct opposi- 
tion to the traditional teaching: the professionals 
place something before the subject in only a fourth 
of their sentences (24. 47 percent); that is, over 15 
percent of their sentences begin with the s ubject. 
Of those opening by something other than the sub- 
ject 23 percent of the total sentences open with ad- 
verbial modifiers, 1.17 percent use verbal groups 
and the inverted Structure appears in only one of 
every three hundred sentences (0. 32 percent ot the 
total sentences). The excessive use of verba] group- 
ings as openers produces tensen 
calls ‘‘pretzel prose“ 

He concludes: 


In another stud 
what he calls ** 


) develops 
tence, ” 


toric of the Sen- 


lation of language, ” 
that of others he dra: 
the rhetoric of the Sentence: 


l. Composition is essenti: 
dee ally a process of addi- 


2. The cumulative 


3. Additions are mad 
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TABLE 1 


MARK OF MATURITY IN WRITING 


Average length Ratio of Average length Ratio of Average 
of clauses Clauses per of T-units T-units per length of 
T-unit sentence sentences 
Grade 4 6. 6 words 1.30 8. 6 words 1.6 13.5 words 
77 percent 77 percent 60 percent 13" percent 80 percent 
Grade 8 8. 1 words 1.42 11.5 words 1.37 15. 9 words 
94 percent 85 percent 80 percent 117 percent 94 percent 
Grade 12 8.6 words 1. 68 14. 4 1.17 16. 9 words 
100 percent 100 percent 100 percent 100 percent 100 percent 


Using visual analysis Christensen shows how sen- 
tences are built (or generated) by the process of co- 
ordination (addition of parallel elements), by subor- 
dination, (addition of levels of modification) or by a 
combination of coordination and subordination. In es- 
sence Christensen recommends and demonstrates 
the superiority of the cumulative sentence over the 
complex sentence. His technique ofanalysis can en- 
rich the study of literature as well as develop ma- 
ture writing. 


Two years later Christensen extended his theory 
of the sentence to the study of the paragraph. Using 
a parallel title **A Generative Rhetoric of the Para- 
graph” (Christensen, 1965), he shows that the prin- 
ciples of analysis he developed for sentences apply 
with equal cogency to the paragraph; that is, para- 
graphs are created by addition, involving coordina- 
tion of elements, subordination of elements, or com- 
binations of the two, As well as addition, there 


1. The paragraph may be defined as a sequence 
Р т а О related sentences. 
i е top sentence of th i i 
tei е Sequence is the topic 
4. Simple Sequence. 
з are of two sorts: i- 
nate and Subordinate, e —SÓ 


6. Some 

апоста арла have no top, по topic, sen- 
The nine headi 

illustration, TI i 

Paragraph ma he conclusion to be drawn is that the 


l unit; Becker calls 


S two 
has three functional аыл, tal 


R (restrictions), and I (illustration). These slots 
correspond to three levels of generality in para- 
graphs (cf. Christensen) and reflect anatural way of 
talking or writing about something. 


The second major pattern has two slots labeled P 
(problem) and S (solution. The P may be aquestion 
or an assertion; the S is the answer or res ponse, 
Where the S is extended, it tends to follow the TRI 
Structure. The majority of expository paragraphs 
fall into these two basic patterns. 


Variations in the patterns are caused by deletion, 
reordering, addition, and combination. The formal 
markers of internal tagmemic structureare graphic 
(indentation), lexical (equivalence classes and lexi- 
cal transitions), grammatical (changes in the gram- 
matical role of equivalence classes signal new slots 
or new paragraphs), and phonological (shifts in pitch, 
register, tempo, and volume in oral reading). 


This analysis makes possible the partitioning of 
paragraphs in a consistent and predictable way. 


The accurate and impartial evaluation of compo 
sition is one of the major problems of English teach 
ing. PaulDiederich (1966) made a study of the fac- 
tors that account for differences in the grades of 
qualified readers. Three hundred papers on two 
subjects were read by sixty distinguished Keane 
representing six different fields: English, Fin da 
Studies, natural Science, writers and editors, f 
yers, and business executives. By comparison 0 
the ratings of three readers who stood highest in a 
particular factor with three who stood lowest in the 
Same factor, it was possible to isolate influential 
factors. In all, 11,018 comments on 3,557 papers 
were tabulated. 


The factors revealed stood in this order of fre- 
quency: ideas, mechanics, organization, wording, 
style or “flavor,” For experimentation in high 
schools, these were divided into two groups: I. Gen- 
eral Merit (ideas, organization, wording, and fla- 
vor); IL. Mechanics (usage, sentence structure, punc- 
tuation, capitals, abbreviations, spelling, hand- 
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TABLE 2 


DISTRIBUTION OF RATINGS 


Non- Academic (N - 476) 


Grade 10 (145) 11 (167) 12 

High 5 percent 8 percent 9 percent 
Middle 34 percent 53 percent 63 percent 
Low 61 percent 39 percent 28 percent 


writing, neatness). A rating card was developed 
for the use of evaluators. A close following of the 
techniques developed by Diederich, with a year of 
practice for a faculty, should yield a reliability of 
the cumulative total for each student close to . 90. 


In the same article Diederich (1966) develops an 
elaborate but practical procedure for the rating of 
large numbers of papers. By a process of sorting 
and grouping of anonymous papers of combined grade 
groups, such as grades 10, 11, and 12, the process 
results in precentage figures for selected groups 
like Non- Academic and Academic with significant 
differences for High, Middle, and Low in eachgroup, 
and a recognizable growth figure from grade 10 to 
grade 12. This extract (see Table 2) from a senior- 


high population will illustrate the distributionof 
ratings. 


In this lower ability groupthe progress in the 
highest level is slight but observable: 4 percent. 
The progress in the middle level is Striking: from 
34 percent to 63 percent; the reduction in the low 
group is equally striking: from 61 percent to 28 per- 
cent, The figures from the Academic group show 
great advances in the high and middle level. This 
procedure demonstrates to teachers, administrators, 
and parents a positive progression of improvement 
not demonstrable by customary paper- grading 
processes, 


RESEARCH IN 
LANGUAGE LEARNING 


Ruth Strickland ( 
structure of childre 
through the sixth gr 
Structure of langu 
teach children to 


1962) undertook to analyze the 

n’s oral language in the first 
ades, to compare it with the 
age found in the texts designed to 
read, and to discover, at selected 
&rade levels, the influence of any ostensible differ- 
ences in the quality of children's reading skill. A 
major hypothesis was that à study of children's 
Speech might offer Suggestions for the construction 
9f better textbooks in reading. 


In the anal si -fi - 
utive sent ysis of structure, twenty-five consec 


ences recorded on tapes were used as the 

de Sample for each child. A Special scheme 
levels s for the analysis of the language at two 
tionary aie vel One was divided into “slots” or sta- 
which ¢ ments, and “moveables, ” or elements 
ап appear in different positions. Level Two 


ммм 


was concerned with the types cf subordinations or 
''satellites'' used in the fixed slots and the move- 
ables. This structure analysis is an important con- 
tribution of the of the study. Tabulations from this 
analysis were programmed to computers to relate 
the frequencies to the variables of age, sex, intelli- 
gence, socio-economic status, and parent education. 
It was found that many similar patterns were used 
with great frequency by children of every grade. It 
was also noted that children of all levels useda wide 
range of language patterns, and could elaborate and 
expand sentences by the use of moveables and sub- 
ordinate elements. 


In the comparison of children's language with that 
of textbooks in reading, four sets of wide ly used 
textbooks were analyzed for language patterns. The 
patterns used by children and by texts at relevant 
grade levels were studied. No scheme for control 
over sentence structure to parallel that over vocab- 
ulary seemed to exist. This raises the question: 
Can knowledge of the structures used by children be 


used to devise textbooks controlling the difficulty of 
reading? 


A study of the relationship between children’s 
use of language and the skill they demonstrate in 
learning to read in grade 6 revealed that children 
ranking high in the variable elements made more 
use of the common oral structural patterns than did 
children ranking low; the same children also made 
greater use of moveables and elements of subordi- 
nation than did children ranking low. A mong the 
conclusions to be drawn are these: 


a. Children’s command of language patterns is 
much broader than is generally realized, 
b. There is almost no relationship between chil- 


dren's oral patterns and the patterns of struc- 
ture in reading texts, 


C. Children can be helped to recognize the pat- 
terns of expression and bythis means be 


aided to advance in reading andinthe learning 
of grammar and usage. 


Walter Loban, of the Univ 
gan in 1952 a longitudinal stu 
in language which continued fi 


ersity of California, be- 
dy of children's growth 
or eleven years, His 


their proficiency in readi 6, teach Да 
judgments of their skill with language, and md ane 


lth and homes, 
| Loban developed what he calls “һе co 
tion unit’’ in analyzing childre 

defined as a group of wor i 

divided without the loss "I1 x v ое ег 
Another term he developed is 
of words or initial parts of wo: 
up to meaningful com municati 
ples recorded on disks and ta 
source materials. Two Sub- 
ally high and one ex i 
ity, were identified. anguage abil- 


mmunica- 


on. 
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A large number of specific findings have been 
summarized in the published report as follows: 


During the first seven years of Schooling, chil- 
dren speak more words in each succeeding year, 
produce more communication units, and increase 
the average number of words in those units. 


Children rated as skillful in language reduce both 
their incidence of mazes and the number of words 
per maze. 


Not pattern, but what is done to achieve flexibil- 
ity within the pattern proves to be a measure of ef- 
fectiveness and control of language at this level of 
language development. 


Children who are rated as most proficient in lang- 
uage are also those who manifest the most sensitiv- 
ity to the conventions of language. 


Those high in general language ability are also 
high in reading ability. . . Writing ability is related 
to socioeconomic position. . . The highest correla- 
tion in the study is between vocabulary and intelli- 
gence . - . A low but positive relation exists between 
health and language proficiency ... Competence in 
the spoken language appears to be basic for compe- 
tence in reading and writing. 


An important aspect of language learning in chil- 
dren is their ability to learn and use standard Eng- 
lish. Loban in Problems in Oral English (1966) 
States that for most children a command of standard 
English is necessary, for *'society exacts penalties 
of individuals who do not possess it. '' In continuing 
his longitudinal study of children's language, Loban 
here examines examples of nonstandard oral usage 
among four different groupsof school children from 
kindergarten through grade 9. His chief findings 
have been summarized thus: 


In ten years of Schooling, pupils from homes in 
which class dialect is used make almost no improve- 


ment in using the verb to be appropriately огіп 
Standardized verb forms. - E : 


The practice, so common among weaker teach- 
ers, of drilling all pupils on the same skill is not 
Supported by this research. Individual pupils, but 
not whole classes of Pupils, will need hel p if they 
are to use the Standard forms of irregular verbs. 


As the complexit 
volume of spoken la: 
more than pro 
*:* Problems of 
of habitua] usage 


y of sentence Structure and total 
anguage both increase, there isa 
rtional probability of difficulty with 
clarity and precision, not problems 


THE EMERGING GRAMMA 
RS 
AND DIALECT STUDY 


Bateman and Zidonis (1966) offer a tentative answer 


blasted by the development of at least two other sys- 
tems of English grammar and the reasonable expec- 
tancy of others to follow. It is no longer accurate 
to speak of the grammar of English except in refer- 
ence to the complete operation of the English lang- 
чаре. We now have several grammars, each a sys- 
tem to describe how the English language works, or 
more specifically, how English makes sentences. 
The traditional system was developed principally in 
the eighteenth century by scholars whose grammat- 
ical disciplines were derived from Latin and Greek. 
Traditional grammar, therefore, was developed to 
make English confor m ло structural principles of 
Latin and Greek. Aside from certain classical bi- 
ases, and lack of then undeveloped linguistic prin- 
ciples, it is a system of analysis of English of Enan 
value in the past and is by no means obsolete at pres- 
ent, unless viewed as the grammar of English. 


In 1952 C. C. Fries introduced structural gram- 
mar. He was chief among a number of sch o1 ars 
working along parallellines. The contribution of 
Structural grammar is accurate description: it is 
concerned with the detailed elements of a language, 
its sounds, word units, inflections, and syntax. In 
the latter, analysis is conducted on the basis of im- 
mediate constituents as operating in modification, 
predication, complementation, and subordination. 
Structural grammar has greatly enriched knowledge 
about the elements of English andthe manner in 
which they are combined to express meaning. 


Noam Chomsky (1957) of MIT brought out in pub- 
lished form a new approach to grammar under the 


title Syntactic Structures. Somewhat modified by re- 


visions, this newest of grammatical theories has 
great promise for the future. Generally called gen- 
erative-transformational grammar, it is a system 
which attempts to state the “rules” which generate 
all English sentences and only English sentences. It 
assumes that we have in us a mechanism for using 
these laws which we gain from intuition. We are 
therefore able to generate sentences that have never 
been uttered before. The analysis begins with ker- 
nel sentences, simple declarative sentences without 
development. From these kernels other sentences 
are derived by means of rules for transformations, 
rules which change or develop kernel sentences соп" 
taining phrase structures which are grammatical 
parts of a grammatical sentence. The goal of this 
new grammar is to state the rulesthat will make the 
production of sentences predictable. 


Applications of structural and transformational 
grammar for classroom purposes are developed oy 
Owen Thomas (1965) and Paul Roberts (1964). The 
former develops the transformational system in easy 
Stages for the English teacher to understand and uti- 
lize the processes of generative- transtor mational 
grammar in the instruction of students. The latter 
presents a set of programmed lessons for self-in- 
struction in the new grammar. N. S. Blount (Blount 
and Johnson, 1966) of Madison, Wisconsin, has cre- 


ated a similar programmed series of lessons adapt- 
ed to the eighth grade. 


The following curriculum question arises: “What 
happens to students who are taught the structure of 2. 
English by Senerative-transformational processes? 


CURRICULUM RESEARCH AND DEVELOPMENT IN ENGLISH 63 


às the result of an experiment with ninth-and tenth- 
grade students. Using equivalent experimental and 
control groups, they taught the former by transfor- 
mational techniques and the latter by traditional 
techniques. A valid sampling of free wr iti ng of 
each student was collected at the beginning and end 
of the instruction. The analysis of these writings 
by the Hunt (1965b) process revealed that the exper- 
imental group wrote better sentences of more so- 
phisticated structure than did the control group. A 
parallel experiment is now under way at Manitowoc, 
Wisconsin, under the leadership of N. S. Blount of 
The University of Wisconsin, Madison. 


A Dictionary of American Regional English is in 
progress, centered at The University of Wisconsin 
and directed by F. G. Cassidy of the Department of 
English (1966). Such a dictionary has been planned 
for over seventy-five years, and active collecting 
has been going on since 1889. Now under a 5-year 
contract with the U. S. Office of Education the work 
is progressing rapidly. Collections of data for six 
states, including Wisconsin are com pleted; six 
more are partially completed. Work of a similar 
nature is well under way in eleven more states. 
Many of the field workers live and work in **Word 
Wagons,” specially equipped small buses contain- 
ing the essentials of living quarters and apparatus 
for field collection. 


Data are assembled, classified, and placed in a 
computer file at Madison. Collectors record speech; 
readers analyze regional literature and local news- 
Papers. The effort is to secure complete coverage 


of the words and expressions used differently in var- 
ious areas of the United States. 


Curriculum applications of this study have yet to 
be worked out, but they will include 1) greater and 
more precise knowledge of American dialects; 2) a 
classification of usage levels and standards; and 3) 
new attitudes toward local and regional language 
variations. 


WISCONSIN ENGLISH 
CURRICULUM ACTIVITY 


By means of a Federal contract initiated in May, 
1963, the State of Wisconsin has been served by the 
Wisconsin- English- Language- Arts Curriculum Pro- 
ject, jointly sponsored by The University of Wiscon- 
sin and the Department of Public Inst ruction. By 
means of two series of bulletins, conferences, con- 
ventions, and personal school visits, more than 
5, 000 elementary- school and high-school English 
teachers have been involved, many very actively. 
The Project is, in fact, a statewide in-service train- 
us Program in the English language arts. The aim 
End encourage teachers to think, confer, discuss, 

RS about the curriculum in En glish. They 
go out sponded magnificantly. Over 3, 000 bulletins 
duplicates month; іп the larger cities these are 
t üchers s and distributed to additional numbers of 
e Project a means of the reports of group leaders 
Chers у irector is informed of the number of 
represent Pd meeting, the grade levels they 
raj » and their views on the issues that are 

In th 
еге тере 8 


ummer о 
Sent f 1964 


a selected group of teach- 


116 all school levels met to produce a 


curriculum in the teaching of literature - a cumula- 
tive growth pattern from the kindergarten through 
grade 12. After some months of revisions and edit- 
ing, this was published in January of 1965 as Teaching 
Literature in Wisconsin, a paper-bound volume of 

160 pages. Nearly 5,000 copies were distributed 

free in Wisconsin; it is estimated that the total dis- 
tribution, including sales at cost, is about 17, 000 
copies. 


By a similar process in the summer of 1965 a 
new group of selected teachers prepared the mate- 
rials for the volume Teaching Speaking and Writing 


in Wisconsin (1966). Over 4, 000 copies were dis- 


tributed free; a large number have been sold at cost. 


Again in the summer of 1966 two curriculum con- 
ferences were held, the first composed of ten selec- 
ted college or university teachers who used their 
linguistic knowledge and experience in the teaching 
of English to plan the language curriculum. This 
was followed by a group of teachers representing el- 
ementary, junior high school, and senior high school 
levels which was to workouta continuum of language 
learning from kindergarten through grade 12. The 
result of this creative work and subsequent editing 
was issued in February of 1967 under the title Teach- 
ing the English Language in Wisconsin. Featured 
in this volume are: 

l. A linguistic approach to grammar, utilizing 
simple applications of Structuralandtransfor- 
mational grammar to the language learning of 
children and youth. 

2. A program that is sequential and поп -repeti- 
tive. 

3. A grammar that provides for more Sophisti- 
cated sentence analysis at the time students 
need it. 

4. An enrichment of the language program by the 
inclusion of word- study, Semantics, and the 
history of the English language. 


The three curriculum 
issued in experimental editions. They were tho r- 
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* Research in Reading 


кшш Wayne Otto and Thomas C. Barrett 


Generally, the 
ive in its ap- 
ns ofagiven 
een a series of 


The present investigation grew out of a recurring 
Concern for pupils' development of tlexibility in per- 
formance in the basic Skill areas. 
titled “Тһе Perce 


writing. The authors pointed out 
might well have implieations for 
skills development, 

again in two studies ( 
rick and Otto, 1961) 
Plication of these stu 
skill such as handwr 
Clated with the devel 
Sure but rather with 


other areas of 
The question of flexibility arose 
Harris and Rarick, 1957; Her- 
of the handwriting act. One im- 
dies was that effic iency ina 
iting does not seem to be asso- 
opment of a fixed level of pres- 
Withi Е the ability to control pressure 
dic in given limits, Another study by Otto (1961) in- 

erage that among good readers in the elementary 
thetic. 7°SPOnsiveness to auditory, visual, kine s- 
» and Combined presentations of stimuli varies 

Th € to grade leve] and type of reinforcement. 
tion variance, however, was not always inthe direc- 
expected, which suggests that one effect of the 


i 
men "че Чопа] Program may be the develop- 
of Perceptual sets in reading. 


The University of Wisconsin 
Madison, Wisconsin 


=== Theodore L, Harris 


University of Puget Sound 
Tacoma, Washington 


These studies led the Committee to the U. 8. Of- 
fice of Education Project No. 1755, “The Ex peri- 


program the chil- 
ef introduction t. 


5 ing speeds for 
purpose approximately 40 Percent, with fu nd 


training; 


Be style 
? d) and dis- 
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according to reading purposes than dotourth graders; 
b) children, by their own daily ratings, greatly en- 
joyed the 2-week training period which was largely 
self- instructional in nature; c) during the short 
training period, children became significantly more 
realistic in predicting their reading speeds when 
asked to read for sequence of ideas, but not for 
main idea or specific fact; and d) no untoward e v i- 
dences of tension were found to be associated with 


reading the test passages or answering questionson 
them. 


In keeping with its past policies, the Committee 
wanted to relate directly its research tasks to spe- 
cific aspects of school practice or to provide the 
basis upon which its findings could be generalized 
to school programs. The experimenters felt that 
before schools could be expected to implement the 
findings of Project 1755 in their programs, they 
Should have empirical evidence that the results of 
training given under the aforesaid experimental соп- 
ditions would indeed transfer to more typical read- 
ing tasks faced by intermediate-grade children. 
The present study is, therefore, an extension of 
Project 1755 specifically designedto investigate the 
possibilities of such transfer effects. 


The first part of the present study replicated the 
training and testing of Project 1755. Theonly major 
revision in the first part of the study was inthe sub- 
stitution of a second active learning group for a pas- 
sive control group. Since the preceding study gave 
evidence that trained groups achieved greater vari- 
ability than controls, it was felt to be more perti- 
nent to test for significant differences betweentypes 
oftraining. Therefore, half of the subjects were 
trained with passages containing anexplicitly stated 
main idea and the other half worked with passages 
in which a main idea was implicit but not stated. 
Classroom materials may or may not contain ex- 
plicit main idea statements So the intent was to de- 


termine whether training with either type would pro- 
duce better results, 


Three major hypotheses were stated: 

1, Inter mediate-grade children trained with pas- 
sages containing either explicitly stated or 
implicit main ideas will differ in r 
ability when reading carefully structured ex- 


pository or narrative materials f i 
t - 
ferent purposes, a 


Inter mediate-grade chi 


Sages containing either explici 
8 S c plicitly state 
implicit main ideas will 4 ie 


IDE transfer their varia- 
bility of i i 
ee reading rate to more typical materi- 


ate vari- 


ldren trained with pas- 


‚апа 6 trained 

wit iei 

main ideas апа children Side Ае 
| а 1 sages 

implicit, but not 


stated, differ in their rate variability when 
reading either expository or narrative materi- 
als for three different purposes on a training- 
like task immediately after training? 

3. Do boys and girls in grades 4, 5, and 6 trained 
with passages containing either explic itly 
stated or implicit main ideas differ in th eir 
rate variability when reading either expository 
or narrative materials for three different pur- 
poses on a transfer task immediately after 
training? 

4. Do boys and girls in grades 4, 5, and 6 trained 
with passages containing either explicitly 
stated or implicit main ideas differ in their 
rate variability when reading either expository 
or narrative materials for three different pur- 
poses on a a) training-like task immediately 
after training; b) transfer task immediately 
after training; c) transfer task one month after 
training? 

5. Do children vary their rate when reading 
either expository or narrative materials for 
three different purposes – specific fact, se- 
quence, or main idea on a a) training-like 
task immediately after training; b) transfer 
task immediately after training; c) transfer 
task one month after training? 


METHOD 


SUBJECTS 


In the final analyses, seventy-two children from 
grades 4, 5, and 6 of a middle-class elementary 
School in Madison, Wisconsin, served as subjects. 
There were twenty-four subjects at each grade level, 
twelve boys and twelve girls. In an attempt to ex- 
clude the extremes of reading ability, subjects were 
chosen who had reading test scores that placed them 
between the fortieth and ninetieth percentiles for 
their grades. The STEP Reading Test was used in 
grade 4, the Reading Comprehension subtest of the 
California Achievement Test was used in grade 5, 
and the Iowa Reading Test was used in grade 6. 
School personnel corroborated the reading test 


scores, Table 1 shows the percentile range by 
grade and sex. 


Within sex and grade level, equal numbers of sub- 
jects were randomly assigned to the two treatment 
groups. To guard against attrition due to absence, 
inability to cope with the task, and examiner errors 
extra subjects, one boy and опе girlfrom each gra ds 
level, were included in each treatment group. ps 
fore, the treatment groups comprised а in I= 
children, of whom six were not included in the ana 
ysis. The extra subjects remaining after attrition 
were randomly dropped. 


EXPERIMENTAL TREATMENTS! 


The distinction between groups was in 1) the na- 
ture of orientation to reading for the main idea and 
2) the structure of materials read during the orien- 
tation and training periods. During these periods, 
Group I worked with materials? inwhich a main 
idea was explicitly stated; the task in reading for 
the main idea was to identify the main idea from a 


list of sentences taken from the passages read. 


y 
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TABLE 1 


PERCENTILE RANGE OF SUBJECTS' READING TEST SCORES BY GRADE AND SEX 


Grade 
Fourth Fifth Sixth 
i i Lowest 
igl Lowest Highest Lowest Highest 5 
Ѕех Rew А Percentile Percentile Percentile Percentile Percentile 
Boys 88 48 90 62 87 40 
Girls 90 50 90 54 89 57 
Total 90 48 90 54 89 40 


Group II worked with the materials from which the 
stated main idea had been deleted; the task in read- 
ing for the main idea involved formulating a main 
idea and then selecting the main idea sentence from 
a list identical to the one used by Grou pl. Both 
groups worked with passages of identical length and 
essentially the same substantive subject matter. 


MATERIALS 


Materials for orientation to the task, for varia- 
bility training, and for immediate post training eval- 
uation were taken, with but minor adaptations ex- 
plained below, from materials constructed for Pro- 
ject 1755. Longer materials, more representative 
of the type encountered by elementary pupils, were 
constructed for use in the attempts to assess the 
transfer effects of the variability training. The de- 
velopment of materials was a major focus in both 
the present project and the one that preceded it; 


thus, the materials are described in considerable 
detail. 


ORIENTATION MATERIALS 


The orientation materials comprised three sets 
of passages, one set for each of the three purposes 
for reading (see description of the training program). 
Each passage was printed separately on charts 


ing sentences were expanded slightly to preserve a 

uniform length. Second, an additional paragraph 

was used with Group II during its orientation to the 

purpose of reading for the main idea. This para- 
graph was similar to the first one in topic and con- 

Struction, but it was used to provide further clarifi- 
cation of the task and practice in arriving independ- 
ently at a conception of the main idea. A pilot study 
showed that the added practice was needed. 


TRAINING MATERIALS 


The training materials developed in Project 1755 
and used in the present study consisted of three 
parts; a set of directions, the passages to be read, 
and comprehension checks. Brief descriptions 
of each of the three parts of the training materials 
follow. 


Directions 


The primary pur 
establish a definite 
find the Main Idea 

e ? ora 
Specific Fact. 


asked 


c d r gned to make the training self- 
Which could be read from the location of any pupil instructional, 
in the testing room. A list of five names with cor- 
» d iE Passages 
Tesponding telephone numbers was used in finding a 
Specific fact, 


Reading for sequence involved one 
basic paragraph with three different orderings of its 
five Sentences. For Group I orientation to reading 
for the main idea also employed a basic paragraph 
Which was rewritten three times so that the topic 
qentence appeared in different parts of the passage. 


р e materials Were taken from those developed for 
Toject 1755. 


The passages were writti 
strictions and consideratio; 
of the training materials w 


en with the follow 
ns in mind. 
as from the 


ing re- 
The Content 
areas of so- 


5 that reading 5 i 
fected by different kinds of content, крес 


The passages were Written in nar i 
I rative and ey. 
Б { А pository styles because а pil i i БЕ 
cludeg ase ац of the Project 1755 materials in- Showed that expository а re Ргојесітт55 
абаран ШУ Stated main ideas, the following i US 
х ns 


rst, th Were required for use with Grou p IL 
> The sentence Stating the mainidea was de- 

Passage used to orient the subjects 

the main idea passage and the remain- 


leted from he 
to reading for 
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selection) or a deductive structure (i.e., the topic 
sentence came near the beginning of the selection). 
The decision to consider structure of the material 
was based upon results of pilot testing in Project 
1755 which indicated that children perform better 
on paragraphs organized deductively than inductively. 


Passages were limited to six to eight sentences 
with total word count ranging from 66 to 112 words 
since the earlier pilot study showed that fourth- 
grade students had much difficulty with longer se- 
lections before training. 


In an attempt to keep the passages within the 
fourth-grade readability level accordingtothe Dale- 
Chall formula, the vocabulary was limited to grade 
4 word lists, sentence length was limitedtoa seven 
to fourteen words range, and typical subject-verb 
word order was maintained within the sentences. 
Each passage was allowed a maximum of two exam- 
ples of the syntactical complication which follows: 
relative and subordinate clauses, participial 
phrases, infinitive phrases separated from the main 
verb, and inverted subject-verb word order. Five 
teachers with experience in the intermediate grades 
judged the difficulty of the materials to be appropri- 
ate. As an added check, the directions for each 
passage encouraged the subjects to ask for vocabu- 
lary help as needed. 


Each passage has a single main idea. Group I 
practiced with passages in which a main idea was ex- 
plicitly stated; Group II practiced with passages in 
which the same main idea was implied but not stated. 
In the passages where the main idea was not stated, 
adjectives, adverbs, and other words which did not 
change the meaning of the paragraphs were inserted in 
the remaining sentences to yielda constant paragraph 
length. The nature of the daily training materials 
is shown in Table 2. 


Comprehension Tests 


The position was taken that the reader should 
have access to all the passage units when selecting 
the appropriate main ideaor determining the proper 
Order of ideas. Thus, for the main idea and se- 
quence tasks the comprehension tests contained a 
condensed or abridged restatement of each of the six 
to eight sentences in the passage itself and the sub- 
ject either identified the main idea statement or se- 
quentially ordered the statements given. For the 
Specific fact tasks, a phrase containing a specific 


fact was taken from each sentence and the subject 
identified the correct one. 


POST- TRAINING MATERIALS 


The directions and со 
Ње Post-training materi: 
the training materials р; 


TRANSFER MATERIALS 


The centr: 
iability qq al question of the study is whether var- 


reading speed developed by training with 


tht "ШШ 


TABLE 2 


NATURE OF THE DAILY TRAINING PROGRAM 
MATERIALS 


Number Selection 

4 Main Idea Selections 

1 Sequence Selection 

1 Specific Fact Selection 


carefully constructed, short model paragraphs Wi 
transfer to reading the longer, less tightly struc 
tured passages typically found in classroom materi- 
al. Thus, the decision was to use the types of ma- 
terial actually employed in classroom instruction in 
the transfer task. With this decision, there was the 
problem of learning more about the nature and length 
of ‘‘typical’’ paragraphs in elementary level materi- 
als. A preliminary survey of tradebooks and text- 
books, especially basal reading series, revealed a 
lack of consistency within intermediate level materi- 
als and even within individual publications. 


It had at first been assumed that construction of 
the transfer materials would involve the use of acer- 
tain number of paragraphs, but the survey revealed 
that there was a surprising lack of correspondence 
between paragraphing and topical units. Within a 
single paragraph, two or three topics might be dis- 
cussed; yet at another place in the same material 
several paragraphs might be used to cover one topic. 
Since one of the purposes for reading in the study 
involved the identification or formulation of a main 
idea, the experimenters decided to select passages 
that contained only one main idea, regardless of the 
total number of paragraphs involved. The survey 
showed also that there was little consistency among 
the materials with regard to the number of words 
used in the coverage of a single main idea. Because 
the experimental design of the study called for auni- 
form length in the transfer test materials, the ex- 
perimenters arbitrarily set the length at 198 words, 


three times the number of words in the post-training 
test materials. 


With consideration for all the factors involved in 
the looseness or tightness of the structure of mate- 
rials at the intermediate grade levels, the experi- 
menters decided that selections from the inter medi- 
ate level Reader's Digest Skill Builders were rea- 
Sonably representative of materials used by children 
in the fourth, fifthe, and sixth grades. 


, Passages were chosen from the Skill Builders in 
view of the following: each passage hadto have a 
Single, complete main idea; each passage had to be 
appropriately Written for one of the three purposes, 
(i.e., if the purpose were to read for sequence, the 
passage must obviously contain a clearly recogniz- 
able sequentia] development); andallofthe passages 
chosen had to be generally interesting to inter medi- 
ate-grade children but not specifically familiar to 
them. The Skill Builder Selections were modified 
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only when necessary in terms of the addition or de- 
letion of a stated main idea, and/or the addition or 
deletion of unimportant words in order to keep the 
length constant. (The authors are indebted to the 
editors of Reader’s Digest for permission to use 
their materials and for their examination and ap- 
proval of all adaptations made in the mater ials.) 
Half of the selections chosen were written in a pri- 
marily narrative style and half were expository in 


style. 


Equivalent sets of materials were necessary for 
the immediate and delayed transfer testing. ‘‘Pairs’’ 
of materials were developed by drawing two selec - 
tions from the same Skill Builder article. Since 
these articles were very short, it was felt that vo- 
cabulary, style, and other factors affecting read- 
ability were likely to be reasonably equivalent. The 
selections in each pair were assigned randomly to 
the immediate or delayed transfer testing materials. 


Table 3 is a breakdown of the total number of se- 
lections used for the transfer study. 


TRAINING AND TESTING PROCEDURE 


In keeping with the framework established by Pro- 
ject 1755, the research program was divided into 
two basic parts, training and testing, which were 
then subdivided with time designated for each part 
as shown in Table 4. 


Training Program 


Training Examiners. A week before the training 
program began, the adult assistants met with the 


graduate research staff for orientation to methods 
and procedures to be used. A brief history of the 
research project was given, and each assistant re- 
ceived a copy of the complete set of directions for 
each day of training. Objectivity in work ing with 
the subjects was emphasized. 


Training Subjects. Days 1-3 of the training pro- 
gram served as an Orientation Period to prepare 
Subjects to read for the three different purposes and 
to familiarize them with the materials and proce- 
dures. The subjects read three practice passages 


each day. The orientation sequence is shown in 
Table 5. 


The difference between groups in the Orientation 


Period was in presentation of reading for the main 

idea. Group I practiced with stated main ideas in 

their paragraphs while Group II practiced with para- 
graphs with implicit main ideas. This distinction 
Was maintained throughout the training program. 


„Оп days 4-10 the subjects practiced for 30-45 
minute periods each day, during which they read 
Six selections daily. The selections were randomly 


Ordered for the tota] group of eighty-four subjects 
every day. 


mu addition to reading passages, subjects also 

Poser пхае tag their reading speed for each 
n. is w : 1 

Speed on their as followed by recording actua 


heete. Tp individual graphs and on Time Record 


Was felt that this would help each subject 
e л 
valuate his own Progress; the graph would enable 


TABLE 3 


NATURE OF THE SELECTIONS USED IN THE 
IMMEDIATE AND DELAYED TRANSFER SESSIONS 


Selections for 
Immediate Transfer 


Selections for 
Delayed Transfer 


Stated Main Idea 


2* Main Idea 2 Main Idea 

2 Sequence 2 Sequence 

2 Specific Fact 2 Specific Fact 
Unstated Main Idea 

2 Main Idea 2 Main Idea 

2 Sequence 2 Sequence 

2 Specific Fact 2 Specific Fact 


*There was one narrative and one expository 
selection in each pair. 


him to visualize the differences in reading rate for 
each of the three purposes for reading. 


On each of the seven practice days, this proce- 

dure was followed: 

1. Each subject received his set of self- instruc- 
tional materials and read the directions to 
establish a purpose for reading. 

2. Each subject predicted his reading speed on 
his graph. 

3. Each subject timed his reading 
with a stop watch and recorded 
time on Time Record Sheets. 


4. An answer sheet accompanying the selection 
was then completed. 


5. Subjects were expected to read to full 
hension or 100 percent 


to the comprehension t 


of the selection 
actual reading 


compre- 
accuracy in responding 
est items. Due to the 


anslated into words 


1 subjects, wh 
supplied with conversion tables, ae 


ADMINISTRATION OF THE TRAINING PROGRAM 


To obtain manageable grou ini 
sions, Groups I and IT were x vider ee лара 
training groups of twenty-one su bjects ЧО; iso 
groups met Separately in the testing r Con d е Ӯ 

the morning, and one £roup met in the afternoon = 


Le? 
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TABLE 4 


SEQUENCE OF TRAINING AND TESTING 


P Testing Program 
Training Program 


ini i ransfer Delayed Transfer 
i i ini Post- Trainin Immediate Trans MEAE 
SERM pos. Test Period. Test Period Test Perio 
erio 
Т Days 14-15 

Days 1-3 Days 4-10 Day 11 Dye 12-19 (one month 

following 

Days 12-13 
TABLE 5 
ORIENTATION SEQUENCE 
Day 1 Day 2 Day 3 


(1) three purposes for reading 
(2) use of stop watches 
(3) use of Time Record Sheets 


Introduction to 


Introduction to 
(1) Conversion tables 
(2) Graphs 


Introduction to Г 
(1) predicting reading speed . 


available at each sessi 


work, 


four or five subje 
of the work and i 
hension tests, 


The followin 


terne 
1. 


2. 
3. 


- One adult w: 


- Instructors 


- Detailed 


on to supervise the subjects’ 
ing ratio of one adult for every 
cts insured adequate supervision 
mmediate feedback on the compre- 


- The result 


£ administrative de 
d after Project 1755: 
Training grou 
one subjects, 
A large, well-ventilat 
TOOm was used, 
Individual work s 
pupil. 

All necessar 
researchers. 


tails were pat- 
PS were no larger than twenty- 


ed, well- lighted, quiet 
Pace was available for each 


y supplies were provided by the 


as assigned to work with every 
four or five Subjects, 
and assistants were trained in the 


testing Procedures and to anticipate possible 


problems, 


Printed instru, 


ctions were provided 
uctors and as 


for instr Sistants, 


TESTING PROGRAM 


usual six selections of varying length. Four ran- 
domly ordered selections were read for each of the 
three purposes. The total reading time in seconds 


for each purpose was recorded. The pupils ids паз 
use their graphs. (2) The Immediate and De They 
Transfer selections were 198 words in length. "m 
are described in detail in the materials ая 2 ot 
this paper. Al] subjects read twelve random ie 
dered selections, six with and six without an waa 
Citly stated main idea, for Immediate UL d 
days 12 and 13. One month later, on days as 
15, they read twelve selections comprising the З 

опа half of the set of transfer materials. During 


the two transfe testing periods, the assistants re- 
corded the subj 


main ideas would а 


materials with im- 
plicit main ideas. 


T 


9 assess the overall effect of 
the treatment conditi 
concerned with the p 


ons, the experimenters were 
est performance On both types 
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of materials. Inordertogeta score which indicated 
this total performance, regardless of type of mate- 
rial, the mean score of the subjects’ performance 
on narrative and expository material was fo und а 
thus reducing the number of scores for each subject 
to six. 


DESIGN AND ANALYSIS 


In order to determine the variability of reading 
rate for the three different purposes – specific fact, 
Sequence, and main idea — a variability score was 
calculated for each subject. The style of the mate- 
rial was considered in that two rate variability 
Scores were calculated for each subject - one for 
narrative and one for expository materials. The 
variability scores were calculated according to the 
Standard variance formula: 


Xi was the sum of individual Scores where i varied 
across the three purposes; X was the mean of the 
three purpose Scores, апі п = 3. Thus, the vari- 
ance scores took into account the subjects’ reading 
rate for one purpose in relation to all pur poses 
(mean rate). There were three different des igns 
and consequently three different types of analysis. 


One was a subject х purpose one way analysis of 
variance design. The dependent variable for the 
analysis was the proportion of each individual's 
reading rate for each Single purpose to his total 
reading rate for all three purposes. 


Due to the fact that each subject read for three 
purposes, a repeated measures situation existed. 
Greenhouse and Geisser (1959) have pointed out that 
for the usually computed F ratios of mean squares 
in the regular analysis of variance tobe exactly dis- 
tributed as the F distribution, it is necessary that 
the columns or tests in addition to be ing normally 
distributed, having equal variances, should be mu- 
tually independent. Since the measures were made 


on the same subjects, the independence assumption 
Could not be met. 


The adjusted de- 
land 71 respec- 
ne using this de- 


grees of freedom are in this case 


tively, Six analyses were also do 
Sign and procedure, 


The | assumed to be fixed, 
iability scores were transformed by 


1 log to base е, Thus, the Scores 
€ analysis were: 


(Ki - х)? 


Hus 


y- 3 
= nat log, У 
1=1 


Из. 


Айег the scores were transformed, an analysis of 

variance was done on the means of the variances ac- 
cording to the procedure outlined by Scheffé (1960) 
for the comparison of variances. Six analyses were 


done according to these design and analysis pro- 
cedures. 


The third design wasa3*2*2«*3 analysis of 
varlance with three grades, two Sexes, two training 
procedures, and three repeated measurements or 
tests — a training test immediately after training, a 
transfer task immediately after training, and a de- 
layed transfer task one month after training. The 
dependent variable for the analyses using this design 
was rate variability. Two analyses were done using 
this procedure. Conservative F tests using the 
Greenhouse and Geisser (1959) procedure were also 
used in this analysis. The adjusted degrees of free- 
dom were 1 and 69 respectively. For all F ratios 
found to be significant post hoc comparisons among 
the means were done following the procedure s of 
Scheffé (1953), 


RESULTS AND DISCUSSION 


The subjects consistently varied their rate when 
reading either narrative or expository material for 
three different purposes. The analyses summarized 
in Table 6 also show that there was a difference in 
rate variability on the immediate post training test, 
the transfer task, and the transfer retention task. 


Mean proportions of reading time are shown in 
Table 7. The trend there is cle 
exception, consistent: 
ly to find specific f 


lapsed across 5 
ence between th 
(Expository - 
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TABLE 6 


ONE WAY ANALYSES OF VARIANCES FOR TOTAL READING RATE FOR THREE DIFFERENT PURPOSES 
WITH EXPOSITORY AND NARRATIVE MATERIALS ON POST, IMMEDIATE, AND DELAYED TESTS 


Description df MS F 
Purposes on Expository Posttest 2 ( 1) .40 40* 
Error 213 (71) .01 

Purposes on Narrative Posttest 2 ( 4) .12 "2* 
Error 213 (71) .01 

Purposes on Expository Transfer Test 2(1) .9T xis 
Error 213 (71) .01 

Purposes on Narrative Transfer Test 2 ( 3) .92 82* 
Error 213 (71) .01 

Purposes on Expository Retention Test 2 (1) .84 84* 
Error 213 (71) .01 

Purposes on Narrative Retention Test 2(1) .36 36* 
Error 213 (71) .01 

#p<.01 

TABLE 7 


MEAN PROPORTION OF READING TIME FOR TOTAL COMPREHENSION BY STYLE OF MATERIAL, 
TIME OF TEST AND PURPOSE 


Purpose 
Test and Style Main Idea Sequence Fact 
Post-Expository 331 „291 
Post-Narrative j 389 à pee „218 
Immediate Transfer-Expositor 8 : io 
1 po y 59 .435 
mmediate Transfer-Narrative .348 .438 4214 
Delayed Transfer-Expository .325 446 .229 
Delayed Transfer-Narrative . 382 С 366 ‚252 
› 
TABLE 8 
| 
MEAN Р 
ROPORTION OF READING TIME BY STYLE AND PURPOSE 
! 
á Purpose 
tyle of Material | 
Main Idea Sequence Fact | 
Exposito 
Эту, 
Narrative .338 .430 252 x 
.373 .399 :228 


B 
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TABLE 9 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR EXPOSITORY MATERIALS AND 
POST PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation 


& 
a 
ы 


5. 98 2.89 
Grade а 1. 00 <1 
Sex 1 55 <1 
Treatment 1 2.99 1.46 
" с хз 2 4.30 2.10 
SAT : 1.14 <1 
SxT - !84 <1 
Санат 2.05 
Еггог 60 
TABLE 10 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR NARRATIVE MATERIALS AND POST 
PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation 


а 
E 


3. 76 1.52 
Grade 2 2.07 «i 
Sex T 1. 66 
Treat 1 4.11 E 
ment 2 . 64 «1 
Gxs 2 225 <1 
GXT 1 1.17 ul 
" SEY T 2 1.24 = 
2.48 
Еггог 60 


The type of materials used during the training The same was true for the transfer rete ntion 
Sessions did not make a difference inthechildren's 


task. When children were te; 
subsequent variability rate when reading for three the transf 
different purposes (see Tables 9- 14. Children 
trained with pass 


However, one might speculate that if the Approach 
ге : i to instruction had been varied for each t of 2 
Specifically Stated main ideas. The types of train- terials, the results might have been different For 
ing materials are subsequently referred toas stated example, if the children who Worked with unstated 
and unstated, respectively. The implication is that materials had been trained explic itly in devis in 
materials varied on this aspect do not affect chi]- their own main idea statements, they might have 
dren’s ability to vary their reading rate, Thus, done better than the other group when all subjects 
either type of training materials should produce sim- were tested on both stated and unstated 
ilar results if reading variability across purposes 


А grams for the develop 
The use of different types of training materials 

did not have any effect on the transfer task. When 
Children 


Were given an extended passage which was 
more close} 


narrative materials, the results were the same " 
y related to the actual School reading Consistently on the Post, immediate transfer, andde 
task, children trained with stated materials did not layed transfer tests no differences were found between 
iffer in their rate variability from children the grouptrained with Stated main idea mai 
trained with unstated materials. 


tated main idea materials, 
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TABLE 11 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR EXPOSITORY MATERIALS AND 
IMMEDIATE PERIOD READING VARIABILITY FOR THREE PURPOSES 


F 
Source of Variation df MS 

Grade 2 4. 94 2. g 
Sex 1 4, 22 d 
Treatment 1 .34 # ^ 
GXS 2 2.35 1 
GXT 2 2.40 1.29 
SxT 1 4. 45 * 1 
GXSXT 2 2. 40 . 
Error 60 1.86 

TABLE 12 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR NARRATIVE MATERIALS AND 
IMMEDIATE PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation 


df MS F 
Grade 2 ‚11 zi 
Sex 1 5. 33 1. 60 
Treatment 1 . 00 <1 
Gxs 2 1.51 <1 
GXT 2 5.10 1.53 
SXT 1 .32 <1 
GXSXT 2 . 03 oa 
Error 60 3. 33 


Th 

Жш = rr nid ae grade level. There 
і mong fourth, fi i 

&raders in reading rate variabili: S fru 


flexible men 


mi 
Ore Susceptible to training 


The ab 


E 
dicates t nce 


of grad 
hat the two 206 


by treatment i 
1 nteraction in- 
9 different training procedures 


EA 


did not have different effects across grade level. 

This might not have been so if materials had been 
prepared for each specific grade level, Further in 

vestigation that focuses upon possible deve lopmenta 

trends might be worthwhile. 


The total reading variability across occasions 
(tests) on expository materials wa s found to differ 
by grade (Table 16). No grade differences were ар" 
Parent, however, on the total reading variability 
across occasions on narrative materials (Table 17). 
Mean rate variability by grade and type of material 
is shown in Table 18, Scheffé post hoc tests for 
multiple comparisons revealed that the only signifi- 
cant difference is between the mean scores of fourth 
and sixth graders when reading expository material. 
This supports the trend found with individual tests. 
By way of interpretation, the suggestion is that older 
Pupils may have a fixed set for approaching the read- 
115 task; whereas, younger pupils may be more 
tadily Susceptible to rate variability training. If 

to be ко, the implication is that such training ought 
© be given early in the intermediate grades. 


t 
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TABLE 13 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR EXPOSITORY MATERIALS AND 
DELAYED PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation 


& 
= 
a 
+ 


Grade 


2 2.97 1.40 

Ѕех 1 .16 EM 
Treatment 1 „21 “д 
Сх5 2 .23 <1 
GXT 2 .31 «d 
SxT 1 E <1 
GxSxT 2 1. 93 «x 
Error 60 2.12 

ТАВІЕ 14 


ANALYSIS ОЕ VARIANCE ОЕ GRADE, SEX, AND TREATMENT FOR NARRATIVE MATERIALS AND 
DELAYED PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation 


e 
& 
= 
n 
= 


Grade 


2 5. 80 2.10 
Sex 1 1.78 <1 
Treatment 1 . 15 Ed 
Gxs 2 2.95 1.07 
GXT 2 2.45 <1 
x 1 7. 
GXSXT 2 2 Л "ram 
Error 60 2. 16 n 
TABLE 15 
MEANS FOR EACH GRADE ON EXPOSITORY ANDN 
ESOS SONS ARRATIVE MATERIALS ON THREE DIFFERENT 
Type Occasion Grade Grade 
4 5 аша 
| Post 3.07 
Expository Immediate 6.29 С 2 2. 08 
Delayed 6. 40 5. 89 5. 46 
я 5.12 
j Post 3.45 
N Л 
arrative Immediate 5. 66 E = 2.12 
D E 1 
elayed 5.80 5.61 5.59 
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TABLE 11 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR EXPOSITORY MATERIALS AND 
IMMEDIATE PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation df MS F 
Grade 2 4. 94 2. 66 
Sex 1 4.22 2.27 
Treatment 1 .34 <1 
G*s 2 2.35 1. 26 
GXT 2 2.40 1.29 
SXT 1 4. 45 2.38 
GXSxT 2 2.40 1,29 
Еггог 60 1.86 

TABLE 12 


ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR NARRATIVE MATERIALS AND 
IMMEDIATE PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation df 


MS F 
Grade i E 
" : 5.33 2 
Treatment : B E 
Trea 2 1.51 : 
vu - 5.10 1.53 
SxT : 702 a 
GXSXT : зз | 
Еггог v = 

Sex did not make a differe 


к nce in the - 
lability of children trained w dea toe 


types of materials, 


ining, on 
fer retention tag. 1 transfer task or on the trans- 


n h separat 
ever, th Parate testing occasion 4 
Bra es trends indicated by tne siens (Т, How 
istent. Fourth cans (Table 15) 


ntal set, whi 
d ¥ ich 
Ore Susceptible to training 


The abse: 
3 nce 
dicates that the me one treatment interaction in- 
Tent training procedi 
ures 


i 


did not have different effects across grade levels 

This might not have been so if materials had pem 
prepared for each specific grade level. Further n 
vestigation that focuses upon possible developmenta 

trends might be worthwhile. 


The total reading variability across occasions 
(tests) on expository materials was found to differ 
by grade (Table 16). No grade differences меге ар” 
parent, however, on the total reading variability 
across occasions on narrative materials (Table m. 
Mean rate variability by grade and ty pe of materia! 
is shown in Table 18, Scheffé post hoc tests [01 
multiple comparisons revealed that the only signifi- 
Cant difference is between the mean scoresof fourth 
and sixth graders when reading expository material. 
This supports the trend found with individual tests. 
By way of interpretation, the suggestion is that older 
Pupils may have a fixed set for approaching ће read- 
ing task; whereas, younger pupils may be mo p 
readily susceptible to rate variability training. If 
this is so, the implication is that such training ought 
to be given early in the intermediate grades. 


г RESEARCH IN READING 75 


ТАВІЕ 13 


ANALYSIS ОЕ VARIANCE OF GRADE, SEX, AND TREATMENT FOR EXPOSITORY MATERIALS AND 
DELAYED PERIOD READING VARIABILITY FOR THREE PURPOSES 


Source of Variation df MS F 
Grade 2 2.97 1.40 
Sex 1 16 <1 
Treatment 1 27 «1 

М GX*S 2 23 ph 

j GXT 2 31 <1 
SXT Я, 71 <1 
GxsxT 2 1.93 <1 
Еггог 60 2.12 
ТАВГЕ 14 


| ANALYSIS OF VARIANCE OF GRADE, SEX, AND TREATMENT FOR NARRATIVE MATERIALS AND 
DELAYED PERIOD READING VARIABILITY FOR THREE PURPOSES 


| 
us Source of Variation dt MS Ы 
: 
Grade 2 5. 80 2.10 
Sex 1 1.78 «1 
Treatment 1 ‚15 <1 
Сх5 2 2. 95 1.07 
GxT 2 2. 45 <1 | 
S xT 1 7.36 2.67 
aks xT 2 2.44 ©1 
Еггог 60 2.16 
TABLE 15 
MEANS FOR EACH GRADE ON EXPOSITORY AND NARRATIVE MATERIALS ON THREE DIFFERENT 
TEST OCCASIONS 
Type Occasion Grade Grade Grade 
4 5 6 
Post 3. 07 2. 66 2. 08 
Expository Immediate 6. 29 6.20 5.46 
Delayed 6. 40 5. 89 5. 72 
| Post 3. 45 3.35 dn 
Narrative Immediate 5. 66 5. 92 5.59 
Delayed 5.80 5.61 4. 87 
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TABLE 16 


ANALYSIS OF VARIANCE OF GRADE, SEX, TRAINING AND TEST OCCASION FOR READING VARIABILITY 
WITH THREE PURPOSES AND EXPOSITORY MATERIALS 


Sources of Variation 


dí MS F 
Training 1 10 .05 
Sex 1 2.83 1.35 
Grade 2 (1) 13.57 6. 49* 
Occasions 2 (1) 36.23 17. 32* 
TXS 1 88 42 
TXG 2 3.04 1.45 
тхо 2 37 18 
SXG 2 2. 26 1, 08 
S xo 2 1.57 15 
Gxo 4 14 .85 
TXS XG 2 2.51 1, 20 
TXS xO 2 3.02 1.44 
TXGXO 4 1. 62 78 
SxGxo 4 1.49 11 
TXS XGXO 4 1.53 73 
Within (error) 180 (69) 2.09 
*p«.05 
TABLE 17 


ANALYSIS OF VARIANCE OF GRADE, SEX, TRAINING AND TEST О 


CCASION FOR READING VARIABILITY 
WITH THREE PURPOSES AND NARRATIVE MATERIALS 


Sources of Variation 


в 
a 
Е! 


eH 1 . 86 .30 
ee 1 8.61 3.01 
с 2 7.87 э, 15 
"T 2 2. 55 89 
T e 1 1.61 56 
тхо 2 1. 28 .44 
S ха 2 1.70 . 60 
S xo 2 1. 24 43 
схо 2 28 10 
TXSxG 4 .20 42 
TXS xO 2 2. 34 Е 
TXGxO 2 3.62 Ti 
SxGxo 4 3.25 ? 

3 .6T 
TXS xGxo 4 1.9 p 
ile: T E 

The analyses , 

ae reveal а Gece MEN S Tables 16 апа 17 between the post-training and the immediate and de- 
ja S — post- aining агае ability across layed transfer tests with expository material. The 
gra transfer — with exposito е transfer, ang de- suggestion is that not only does the ability to adjust 
teat ang Alva materia] y materials but pot rate to purpose transfer from training passages to 


more naturalistic materials, but also there is an 
T4 E ә : аы В ight 
a significant difference increase in rate variability. The increase mig 


be expected on the basis that the materials in the 


e 
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ТАВІЕ 18 


MEAN RATE VARIABILITY ON ALL TESTS BY 
GRADE LEVEL AND TYPE OF MATERIAL 


Grade 
Type 4 5 6 
Expository 5. 98 5. 65 5. 12 
Narrative 5. 70 5. 93 5.12 


transfer test situations were more nearly like class- 
room materials and less rigidly structured; there- 
fore, the subjects were better able to usetheirrate 
variability skill. 


It must be noted that the main effects of grade 
and test were significant only with expository mate- 
rials. Yet, the trends were generally similar. The 
structure of the expository material apparently was 
more conducive to rate variability than that of nar- 
rative material. Whether this is generally so, of 
course, remains to be seen. 


There was no difference in the total reading var- 
iability of boys and girls. Boys and girlsthuslearn 
the same amount during the training situations. No 
differences were found on any of the interaction fac- 
tors, which indicates that all factors were consist- 
ent in combination. 


The type of training materials used did not result 
in a difference on the total reading variability 
across occasions regardless of whether the children 
were reading expository or narrative material. This 
again emphasizes the similar effect of either of the 
two types of training materials. It may also indicate 
that in the future this type of research ought to be 
concerned with combinations of materials and in- 
structional procedure in order to maximize the ef- 


fect of training. 


TABLE 19 


MEAN RATE VARIABILITY FOR EACH TEST BY 
TYPE OF MATERIAL 


Test 
Type Post Immediate Delayed 
з. ERA o FA — 
Expository 4, TI 5. 99 6.01 
Narrative 5.37 5. 13 5. 43 


FOOTNOTES 


1. The research reported herein was supperted by 
the Cooperative Research Program ofthe 
Office of Education, U. S. Department of 
Health, Education and Welfare. This paper 
first appeared in manuscript form entitled 
“Transfer Effects of Training Intermediate 
Grade Pupils to Adjust Reading Speed to Read- 
ing Purpose, '' Cooperative Research Project 
No. 3137, Committee for Research in Basic 
Skills, The University of Wisconsin, 1966. 


2. An appendix containing the directions and mate- 
rials used can be obtained from the authors 
on request. 


REFERENCES 


Greenhouse, S. W., and Geisser, S. On methods 
in the analysis of profile data. Psychometrika, 
1959, 24, 95-112. 


Harris, T. L., Herrick, V. E., Macdonald, J. B., 
and Rarick, G. L. Experimental development 
of variability in reading rate in grades 4, 5, 
and 6. Cooperative Research Project No. 
1755, Committee for Research in Basic Skills. 
Madison, Wisconsin: Schoolof Education, 
University of Wisconsin, 1965. 


Harris, T. L., and Rarick, G. L. The problem of 


pressure in handwriting. Journal of Experi- 
mental Education, 1957, 26, 115-178. 


Herrick, V. E., Harris, T. L., and Rarick, G. L. 
Perception of symbols in skill learning by 
mentally retarded, gifted, and nor mal chil- 
dren. Committee for Research in Basic 
Skills. Madison, Wisconsin: School of Edu- 
cation, University of Wisconsin, 1961, 


Herrick, V. E., and Otto, W. Pressure on point 
and barrel of a writing instrument. Journal of 
Experimental Education, 1961, 30, 215-230. 


Otto, W. The acquisition and retention of paired 
associates by good, average, and poor readers. 


Journal of Educational Psychol gy 
241-248. “шы Ee Шш, 


Scheffé, Н. A. A method for judging all possi 
contrasts in the analysis of variance. ст 
гіка, 1953, 40, 87-104. CASA 


Scheffé, Н. The analysis of variance. New York 
Wiley, 1959. ore 


Shores, H. J. Reading science materials for two 


distinct purposes. Elementary English 
37, 546-552, 565. mary Snelish, 1960, 


- Strategies for Concept 


Attainment in Mathematics 


IT WOULD be a mistake for the writer, 
being trained in mathematics rather than psychology, 
to do basic research in conceptual learning. On the 
other hand, it is quite appropriate for him to ac- 
quaint himself with the results of experimentation by 
psychologists and to manipulate these results in dif- 
ferent ways to see how they apply in mathematics 
instruction. In the latter Sense, one might say the 
paper has an Operational flavor. That is, it dis- 
cusses ways classroom teachers might attack in- 
struction in a specific concept of mathematics and 
means writers of classroom materials might use 
to insure that provision had been made for learning 


experiences that would leadtoattain if- 
= е inment of a specif 


oe bee iui who faces a mathematics class five 
Ж E pee often finds psychological research 
ie an € help in solving his instructional prob- 
peciit Еее гең, for obvious reasons, must 
Fio esa ose situations that present knowledge 
cere «3 A research psychologist legitimately 
fore еа interest іп pure psychology; he may 
каш: к thoughts about applications of his re- 
fons ш s the excitement of carrying on investiga- 
pub dh 5 particular field of interest leads him to 
ese thoughts aside — temporarily, perhaps. 


Today, however, 


there exists sufficient evidence 
on human learning as 


5 well as numbers of well-trained 
ma ipn teachers 50 that applications can be 
Classroom It is possible today to have 
manipulates Xperimentation that makes use of, or 
learning E. basic psychological research in human 
" “merging among mathematical educators 


Myron F. Rosskopf 


Teachers College, Columbia University 
New York, New York 


and psychologists are plans for studies that will ap- 
ply in a subject matter field the results obtained 

from the intensive study of conceptual learning of 
recent years (that is, since World War II. Every 
discipline has its own subtleties. For this reason, 
Success of such experimentation will de pend upon 
facility of communication between psychologists and 
representatives of other disciplines. 


DEFINITIONS 


CONCEPT 


The word ‘‘concept’’ seems to have as many def- 
initions as there are writers on conceptual learning. 
Recently, a definition appeared that has the v irtue 
of efforts made to relate it to existing definitions. 
According to this recent statement, | 

pedes that is a concept has the following 

attributes: А 

L Psychological meaningfulness А 
П. Intrinsic, functional, or formal properties 
Ш. Abstractness 
IV. Inclusiveness 

V. Generality 


VI. Structure | | 
VII. Function (Klausmeier, Davis, Ramsay. 


Fredrick, and Davies, 1965, p. 3). 
A possible criticism is that the definition introduces 
words which in turn need to be defined. True, but 
it turns out to be easier to define (or describe) these 
new words than it was to define ‘‘concept. "' 


Two aspects of the definition that make it useful 
are its open-endedness - by means of item V, 
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Generality — and its own inclusiveness. Simple con- 
cepts as well as very complex concepts are possible 
of characterization by means of the definition. 


The writer has been using the following definition 
of a concept, probably because it lent itself so well 
to explanations of mathematical concepts. А con- 
cept is “а way of grouping an array of objects or 
events in terms of those characteristics that distin- 
guish this array from other objects or events in the 
universe’’ (Bruner, Goodnow, and Austin, 1956, p. 
275). This suffices for a first level concept, but to 
obtain the complexity that we know, at least intui- 
tively, some concepts have, we need to add to the 
definition cited; and including ‘‘the network of infer- 
ences that are or may be set into play by an act of 
categorization” (Bruner, et al, 1956, p. 244). 


We shall not stop to relate these two definitions. 
Interested readers can go to the references cited 
and see that the two do indeed contain the same ele- 
ments, Instead we shall consider an example of a 
concept in mathematics and show in what way it sat- 
isfies the conditions of the f irst definition stated. 
Consider, for example, the set of all counting num- 
bers, or as they are often called the set of all nat- 
ural numbers, 

DL E E НЧ" 

One can discriminate certain subsets of this set. 
There is the set of all even natural numbers, 


[2, 4, 8, is RR oisi 
and the set of all odd natural numbers, 
ГОА 29, B, cius "Bey ons Т 


For our example of a mathematical concept, how - 
ever, we shall consider that of the set of all prime 
(natural) numbers, 

[:2,:3, BS, Mp dd, 19, cas]. 


When a person can distinguish between prime 
numbers and numbers that are not prime (that is, 
that are composite), then he has a concept of prime 
numbers. He can demonstrate his possession of the 
concept by filling in the next few prime numbers, or 
by devising a test for deciding whether a number 
presented to him is prime, or not prime. Hehas 
grouped the array of objects, the setof natural num- 
bers, so that he can tell the difference between a 
prime number and a composite number. Hehas 
used the characteristics that distinguish between 
prime numbers and other members of the set of nat- 
uralnumbers. Really, the person has formed two 
categories of natural numbers; those thatare prime 
form one category; and those that are co mposite 


form the other category. 


At this stage of his concept of a prime number, 
a person recognizes the formal properties. He cer- 
tainly is working at an abstract level, since he is 
working with numbers, and at a level of generality, 
Since many subsidiary concepts such as factoriza- 
tion, multiplication, and so on are involved. From 
the point of view of structure, the person probably 
has used a conjunction of related concepts to arrive 
at a concept of prime numbers. 


But we recognize more ina concept than just the 
forming of categories according to some discrimi- 
nating characteristic or characteristics. Part of 
the concept of prime numbers is realization that 
their number is infinite, that there is just one even 


prime, that no one has devised a formula for prime 
numbers, and that there are many such o pen ques- 
tions related to this interesting subset oí the set of 

natural numbers. In other words, we are willing to 

accept as part of the concept these inferences drawn 
from the categorization. 


It seems that such associations with the set of 
prime numbers would qualify it for the attribute of 
psychological meaningfulness. There remain the at- 
tributes of inclusiveness and function. Now, inclu- 
siveness is so much a part of specifying a set in 
mathematics that as soon as one is able (that is, 
correctly in the sense of mathematical criteria) to 
talk about the set of prime numbers, the attribute 
of inclusiveness follows. Prime numbers as medi- 
ators occur in prime factorization, calculation of a 
least common multiple, and the prime factorization 
theorem of arithmetic. Hence, the concept has the 
function attribute also. 


Allright. It is possible to relate what mathema- 
ticians call a concept to what psychologists call a 
concept. But a definition, if it really is a definition 
must partition the universe of discourse into those 
things that are concepts and those things that are 
not concepts. A ''definition" that is all inclusive 
might just as well not be stated at all. In short, it 
must be possible to point to something in mathemat- 
ics that is not a concept. But that is easy; any addi- 
tion fact, such as 5 + 4 = 9, is not a concept. 


CATEGORIZING OR CONCEPTUALIZING 


The word ‘‘category’’ was used in the preceding 
discussion. What is involved in categorizing? How 
is it related to conceptualizing? More important, 
how is categorization related to the use of conc e p- 
tualization in applications, or problem solving, in 
mathematics? First, let us analyze another exam- 
ple of a mathematical concept and, second, derive 
from the discussion of the example what seem to be 
characteristics of categorizing. 


Consider the concept of a rational number. Its 
formation starts very early inthe home experiences 
of a preschool child. One-half and one-fourth are 
known for names of parts of things; one-half of an 
apple or an orange or a glass of juice or a candy bar. 
During early schooling, the concept is expanded to 
include written symbols for one-half and one-fourth: 
1/2 and 1/4; 1/3 also is added to the list. However 
the concept is still on the level of these symbols be- 
ing names for parts of things. There seems little 
evidence that a child thinks of 1/2, 1/4, 1/3 as ex- 
emplars of a new sort of number. The teacher may 
call these fractions names for numbers, of course 
but still the concept, the way of categorizing these’ 
numbers, of telling how they differ from other num- 
bers, is in terms of a way of expressing the m ; to 
wit, a numerator, a horizontal bar, anda denomi- 
nator. 


Not until quite late does the instruction reach th 
stage of giving a new name, rationalnumbers t zw 
set of numbers that can be expressedas the ia m 4 
of two integers, the denominator not zero A s 
concept has formal properties; it has been [seu 


ized, one might say, and it will be even more for 
) when a student recognizes 


mal (that is, abstract 
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rational numbers as ordered pairs of integers in 
which the second member is not zero. 


At each level of formality, there are cues that a 
student looks for: 1) one of two congruent parts of 
Something, such as an apple; 22 a numerator,a hor- 
izontal bar, and a denominator; 3) a quotient of two 
integers; and 4) an ordered pair of integers. These 
cues help a student put a number he encounters into 
its proper category, to align it with the concept he 
has formed of the set to which that number belongs. 
Notice, too, that the invention of ways of grouping 
predictive characteristics is involved; for example, 
the numerator, horizontal bar, and denominator. 
The test of the invention is its usefulness, Does it 
effectively categorize numbers? 


Categorizing has for its basis defining character- 
istics. By grouping together things that satisfy the 
defining characteristics of a class, an individual a) 
reduces the complexity of his environment; b) iden- 
tifies the objects in a particular u niverse of dis- 
course; c) reduces the necessity for constant learn- 
ing; d) gives direction to activity; e) orders and re- 
lates classes (Bruner, et al., 1956, pp. 12-13). 


What are the implications for problem solving in 
mathematics? If a student does not classify prob- 
lems he meets, then he must treat each problem as 
an individual. The cognitive strain in such a proce- 
dure is enormous. The student must, perforce, for 
purposes of economy of effort look for distinguish- 
ing characteristics or discriminable attributes of 
problems in order to reduce the strain. That is, he 
must use cues, put the problem in a pre-decided- 
upon category, and bring into play concepts that in 
the past have been effective in Solving problems of 
this particular category. It would seem that a cate- 
gorization of a problem is the first step toward its 
Solution. Only after a problem has been put ina 


certain Category can one proceed with plans for 
Solving it, 


STRATEGIES IN 
CONCEPTUALIZATION 


coordination of the wr ful Planned: calculation and 


mum nu 
Encounters with relevant iate 


EE 


b. To insure that a concept will be attained 
with certainty, regardless of the number 
of instances one must test en route to at- 
tainment. 

с. To minimize the amount of strain on in- 
ference and memory capacity while at 
the same time insuring that a concept 
will be attained. 

d. To minimize the number of wrong cate- 
gorizations prior to attaining a concept. 
(Bruner et al., 1956, p. 54.) 


The preceding definition of a Strategy and a de- 
Scription of the aims of a Strategy is very similar 
to the mathematical definition of a Strategy that is 
associated with the theory of games. However, the 
theory of games Strategy consists of a set of deci- 
sions made in advance. Once the decisions are 
made, no change can be made while **play"' is going 
оп. In contrast, a problem- solving strategy, for ex- 
ample, may be changed in the midst of a subject's 
efforts; such changes are dictated, or should be dic- 
tated, as the subject's knowledge ofthe problem 
grows through his efforts tofinda method of solution, 


TYPES OF STRATEGIES 


“Virtually all the effective strategies for attain- 
ing concepts depend upon the use of some sort of in- 
itial focus,” (Bruner, etal., 1956, p. 63). How 
does the subject perceive the situation? What as- 
pects of it holds his attention? How does he define 
the task to himself? If the task is a problem situa- 
tion, what data seem to be most relevant? How 
does he fit this problem into his concept pattern of 
all types of problems with Which he has had experi- 
ence? After making an initial focus with respect to 
the problem, a subject still faces the necessity for 
decisions Concerning a method of attack. What sort 
of strategy should he decide upon? A ccording to 
Bruner and his associates there are four ideal strat- 
egies: a) simultaneous Scanning; b) successive scan- 
ning; c) conservative focusing; and d)focus gambling 
(Bruner, et al. › 1956, pp. 83-90). 


The simultaneous 
hypotheses to follow 


on memory is great, Initially, he 
ез and tests it to see 
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b) Successive scanning. Essentially, this strat- 
egy consists of testing one hypothesis at a time. A 
hypothesis is tested until a contradiction is reached, 
or until it is clear to the subject that the desired in- 
formation is not forthcoming. Then the hypothesis 
is discarded, and a second is chosen for testing. 
Notice that a hypothesis is tested independent of the 


others. 


The gain in use of successive scanning is twofold: 
1) it lightens memory load and 2) frees the subject 
from the necessity for making many deductions con- 
cerning the other hypotheses. Two disadvantages 
are that some of the hypotheses may overlap, lead- 
ing to repetition of work already done, and that there 
is no control over risk. A subject cannot take 
greater gambles or lesser gambles. Thisisa rath- 
er leisurely strategy and discounts heavily any time 


factor. 


c) Conservative focusing. First of all, a subject 


who is a conservative focuser studies the situation 
to see if there is one key idea. If there is, it is fo- 
cused upon, and all hypotheses are tested against it. 
Suppose he finds a hypothesis that looks promising. 
That is, a hypothesis that impinges on the key idea. 
Now, he chooses those hypotheses that contain the 
same part that the promising hypothesis does. Then, 
one by one, he adds parts of these hypotheses and 
tests the results. The ‘‘conservative’’ enters this 
Strategy at this point. Only one feature of a hypoth- 
esis is added at a time and tested. 


Much information is obtained by conservative fo- 
cusing and redundancies are avoided. The risk of 
securing no information is under control; in fact, 
one always gets some useful information. The only 
real disadvantage is that information which is not 
arranged in orderly fashion might involve a long 
search for the needed data. Thus, part of the suc- 
cessofa subject using conservative focusing depends 
upon how well organized his knowledge is. 


d) Focus gambling. The initial steps taken by a 
subject who uses focus gambling as a strategy are 
the same as those of one who uses conservative f o- 
cusing. But, he is not satisfied with adding one part 
of a hypothesis at a time and testing; he adds two or 
more parts at a time and gambles. The strategy is 
not quite an all-or-nothing one, but it comes very 
close in case the omnibus hypothesis fails. The 
only thing to do then is for the subjectto shift strat- 
egies and fall back on successive scanning. 


However, the gamble may pay off. The omnibus 
hypothesis may lead to attainment of the objective. 
There is much gain then, of course, for the number 
of trials has been reduced to a minimum and much 
time has been saved. 


THE STRATEGIES IN USE 


IMPORTANCE OF INITIAL FOCUS 


Several years ago, the writer did a pilot study in 
which about ninety secondary- school teachers were 
involved. Two tasks, or problems, were presented 
to the subjects, and they were asked to find a solu- 
tion. The objective at the time was to secure infor- 


mation on these teachers' conceptof a mathematical 
Solution, but other interpretations of the evidence 
turned out to be more interesting. Thisis true, par- 
ticularly in view of the foregoing discussion. 


Situation 1. One problem was the following: 
From a two-digit number, N, subtract the 

number whose digits are those of N inter- 

changed. If the difference is a positive cube, 

how many permissible values of N are there? 
However innocent this problem appears, it is not a 
run-of-the-mill problem. The method of approach, 
that is, the initial focus, is critical. Practically all 
of the teachers classified the problem as a ‘ digit 
problem." Allright; there are standard methods 
for solving a problem so categorized. These were 
applied; let t and u represent the tens’ and units’ 
digits, respectively; then 

N=10t+u 
а positive perfect cube = (10t + u) - (10u + t). 


Quickly perceived is that the tried-and-true meth- 
ods fail. There is no place to go from this point, 
for at least two more equations are needed for them 
to be successful. A small number of papers were 
discarded because of errors in finding a simpler 
form for (10t + u) - (10u + t). It is doubted that this 
was failure of any strategy. Rather, such errors 
seemed to be the result of haste or carelessness. 
The remaining papers fell nicely into three groups. 


There were those whose initialfocus was all 
right. These papers showed that the persons got as 
far as 


N=10t+u 
x3- 9(t - u) 
x 0. 


But, unfortunately, once they had made a focus, they 
were not flexible enough to change it when it led no- 
where. The papers of these subjects were covered 
with equations and tag ends of Scribblings. They 
formed only one hypothesis - secure enough equa- 
tions to solve for all of the unknowns. Опе m ight 
categorize their strategy as successive scan ning 
where there was just one hypothesis to scan. This 
was done over and over by members of this group. 
There was no change of focus with res pect to the 
problem. Failure did not discourage them from try- 
ing the same hypothesis again and again. 


The second group's papers showed fewer Scrib- 
blings. There was hardly any writing on these pa- 
pers after the initial symbolizing of the problem sit- 
uation. Inferred from what evidence was available 
was that these subjects realized that the initial cat- 
egorization of the problem as a d igit problem was 
not quite correct. They seemed tobe flexible enough 
in their focusing on the problem to change their 5 
cus. It seemed, however, that they were at a 1o o- 
to formulate another hypothesis or another strat м 
А possible interpretation is that these people ad 
limited concept of a mathematical Solution. Се, a 
ric proofs and the results obtained by Solvin omet- 
tions qualified all right, but the result iouid c equa- 
and error or by a verbal argument did not trial 


Members of the third grou i 
TS с pnot onk 
a shift in initial focus was necessar ees 
had a better conception of what con: 
matical solution. Some ado pte 


ed that 
y but they also 
Stitutes a mathe. 
d àstrategyor 
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successive scanning, trying different pairs of num- 
bers in succession. Others clearly used conserva- 
tive focusing, for they not only tried pairs of nu m- 
bers but also took into account the positive perfect 
cube datum. Those with the most elegant solutions 
used a highly sophisticated conservative focusi ng 
strategy. А few used focus gambling, but more due 
to the pressure of time rather than to preference. 


Situation 2. The second task presented to these 

teachers was a straight-forward geometry problem: 
Find, on a straight line, a point such that 

the sum of its distances from two given points 

is the least possible. 
The first step in any geometry problem is to draw a 
figure and introduce some notation. If more sub- 
jects had followed this advice, there would have been 
more correct solutions to this problem. As it was, 
only one-sixth of the subjects were able to arrive at 
а solution. Just as was the case with the algebra 
problem, here, too, the difficulty was inflexibility 
With respect to the initial focus. 


Let us take the key idea of the problem, the sum 
of the distance from two given points is the least 
possible, and use it as an initial focus. Now,as one 


proceeds to make a figure and introduce notation, he 
must give 


LJ > 
T A В 
Саѕе 1 


up this initial focus for a moment and shift to focus- 
ing on the two given points. The problem says noth- 
ing about placing them; a decision is г equired. 

Three possibilities, or hypotheses, appear: Case 1, 


both points are on line 2 ; Case 2, the points are on 
Opposite sides of 2 $ 


Ae 


eB 
Case 2 


and Case 3, the points are on the same side of £ , 


For 
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triangle BDC is congruent to triangle B'DC, and 
consequently BD = B'D. Therefore, BD + DA is the 
least possible distance. 


These two problems have emphasized the impor- 
tance of initial focus. More than that, discussion of 
attempts at solutions made clear how necessary it 
is for a problem solver to be flexible enough to shift 
his focus as the process of solving the problem de- 
velops. In each problem, concepts related to th at 
involved in the problems themselves were very im- 
portant. It seems that, at least in mathematics, 
knowledge of these subsidiary concepts is crucial to 
Success. The strategies used seemed first to be 
that of simultaneous scanning with a transfer to con- 
servative focusing after a promising **lead'' had 
been discovered. 


SIMULTANEOUS OR SUCCESSIVE SCANNING ? 


The next problem situations are relatively inde- 
pendent of initial focus, depending much more upon 
a choice of strategy. The mathematics curriculum 
puts considerable emphasis today ona search for 
patterns. The following situations are of that sort. 


Situation 3. Divideyour paper into three columns. 
In the first column there is a String of letters, and 
in the second column there is another String of let- 


ters. Now, we shall use these two strings of letters 
to make a new set 


K,L,M,T,W,Z | L,M,T,x,Y 


of letters in the third column. How were the letters 
in the third column 


K, L,M,T,W,Z | uu m xv L,M,T 


obtained? Now, try to write the set of letters that 
goes into the third 


A,C,E,P,Q | A,C,L,P,Q,R,8 


column for these two sets. What goes into the third 
column for the following two sets? 


T,U,V,W | T,U,v,w 


Here is another example, 


What would you write in 
the third column? 


S,A,M,E NOT 


Is it true that, given sets in the 
columns, there is alwa 
third column? 


first and second 
ys exactly one way to do the 


What are the results if we interchange th 
: es 
in the two columns? Fill in the third Column е 
compare the results with what you got before 


L,M,T,X, Y K,L,M,T,W.Z 
A,C,L,P,Q,R,S AC,E,P,Q' 
T,U,V,w T,U,V.w 
N,0,T S,A,M,E 


What did you find? 
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Situation 4. Now let us form a third column set 
in a different way. As before, we start with sets in 
the first and second columns. How was the 
A,C,E,G,H,J B,C,D,E,F,G,T | A,H,J 


third column set obtained? Now, try to write the 
third column set, starting 
F,A,M,LL,Y E, A,R, L, Y | 


with these two sets. Let us try two more special 
situations. Is it always 

,G 

,D 


possible to obtain exactly one set, given sets in the 
first and second columns? 


A,B,C, D,E, 
A,B,C,D A,B, 


Let us see what the results are if we interchange 
the sets in the two columns. Compare the answers 
with those you had before. What can you say? 


B6,D,E,. FG T | AG, BiG) HJ 
E,A,R,L,Y F,A,M,LL,Y 
D,E,F,G A,B,C 
A,B,C,D A,B,C,D 


We have worked with strings of letters in two dif- 
ferent ways. That is, we used two different pat- 
terns to obtain the set in the third column. Compare 
these two patterns. Do they have the same proper- 
ties? How are they different? 


DISC USSION OF SITUATIONS 3 AND 4 


The last two problem situations are quite differ- 
ent from the first two. Except that they were not 
quite straight-forward, the first two problem situa- 
tions are representative of the mathematics program 
before World War П. The last two are representa- 
tive of types of problems that arise in the new math- 
ematics program. The algebra and geometry prob- 
lems have for their objective getting an answer. Af- 
ter the answer has been obtained, there is little one 
can do with it, except perhaps to refine the method 
of solution. It is not so much that a concept is in the 
process of being attained as it is that pre- estab- 
lished concepts serve as mediators in solving each 
problem. Consequently such problems would be rea- 
sonable to present only after there was some ev i- 
dence that subjects had been at least exposed to the 
necessary mediating concepts. 


On the other hand, the third and fourth problem 
situations are representative of situations intended 
for seven- to nine-year-old subjects. The intent is 
that through such situations a foundation is laid for 
attaining rather deep mathematical concepts – those 
of closure and commutativity in this case. A battery 
of situations of this sort can be devised that has a 
bearing on ordinary arithmetic. Of course, letters 
would not have to be used. One could use the boys 
and girls in a class, or colored counters, or books, 
or any other sort of material available. An effort 
would be made to use materials appropriate for the 
age level of the class. 


It is not clear to the writer that mediating con- 
cepts are brought into play. Rather, it seems that 


solving these problem situations requires use of a 

strategy, one that is effective in attaining concepts. 

Pattern-problems such as these seem to yield to a 

successive scanning or a simultaneous scanning 

Strategy. Or, one uses a combination ofthe two 

Strategies; simultaneous scanning for gross elimi- 
nations of parts of the strings of letters; and suc- 
cessive scanning for determination of fine discrimi- 
nations. 


TWO ITEMS OF RESEARCH 


Important to the mathematics program is knowl- 
edge concerning what types of data in problem situ- 
ations present difficulties. Such knowledge influ- 
ences preparation of classroom materials in two 
ways. Learning experiences can be provided t hat 
are intended to facilitate attainment of a concept, 
and information gleaned from research studies can 
be used to schedule concepts according to age levels. 


An attempt by one person to report on all of the 
research studies concerned with problem solving 
would be foolhardy. A bibliography is very long. 
Two doctoral studies have been singled out for re- 
porting. One reason is to underscore the fact that 
much relevant research gets lost or overlooked 
through non- publication in journals. 


POST STUDY 


In 1958, Post did an experimental study of six fac- 
tors involved in understanding problems. His sub- 
jects were fifth- and sixth-grade pupils in schools 
in the Metropolitan New York section of the country. 
The definition of a problem inthe study was ‘‘a quan- 
titative situation described in words ... but in which 
the arithmetical operations that lead to the answer 
to the question are not explicitly indicated" (Post, 
1958, p. 15). The six factors were: 1) sizeof num- 
bers; 2) superfluous numerical data; 3) familiarity 
of setting; 4) number of steps; 5) type of operation; 
and 6) symbolic terms. 


Factor 4, number of steps, requires some words 
of explanation. Post defined a two-step problem 
and a three-step problem as follows: 

A step in a problem solution... was a 
simple performance of one of the four oper- 
ations of addition, subtraction, multiplication 
and division. Therefore, a problem is (a two- 
step problem) if and only if the solution of the 
problem requires a single performance of two 
different operations or two performances of 
the same operation (Post, 1958, p. 30). 

A three-step problem is defined in a similar way 


Factor 6, symbolic terms, refer i 
of whether using a name for anu и 
"'twenty-six"' in place of “26” interferes witha | 
ject’s ability to solvea problem. The name а 
the other factors are self-explanatory. eo 


Two forms of a test were prepared ; 
forms were judged өйшүшеп! ty a 1 e pes these 
and then tested further in a Pilot study. ШАГ, 
of setting and non-familiarity of Setting was UU. 
from an inventory of such Settings suggested ned 
mentary-school teachers. This infot mator o any 

s 
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supplemented with evidence obtained from the pilot 
study pupils. 


The study design was such that an analysis of var- 
iance technique could be used. The error term 
‘‘was composed of the unconfounded four- and five- 
factor interactions as well as the six-factor inter- 
action” (Post, 1958, p. 85). The two forms of the 
test were treated as two replications of the same ex- 
periment, for examination of the test results from 
the two forms led to doubt that the data fro m them 
could be combined. The schools represented two dif- 
ferent socio-economic neighborhoods; the subjects 
were chosen from non-homogeneous classes; and 
small sample theory analysis was used. Correct- 
ness of solution was determined from examination 
of the work written by a pupil rather than from the 
Correctness of the answer. For example, a pupils 
solution might be categorized as correct even ifhis 
answer was incorrect. Another way of saying this 


is that errors in arithmetic computation were dis- 
regarded. 


A hypothesis was rejected if the re Sults from 
both forms of the test indicated rejection; it was ac- 
cepted, similarly, if both forms indicated accept- 
ance. No conclusion could be stated if one form in- 
dicated rejection and the other for m indicated ac- 
ceptance. 


The hypotheses on the factors of size of numbers 
and number of steps fell in the no conclusion cate- 
gory. So far as the latter (number of steps) is con- 
cerned, it might have been that there was a strong 
interaction between number of steps and type of op- 
eration, for the hypothesis concerning the latter was 
rejected at the one tenth of one percent level of sig- 
nificance. That is, problems involving multiplica- 
tion and division were found to be more difficult for 


the sample subjects to understand than problems in- 
volving addition and subtraction. 


á Hypotheses rejected were those со 
uous numerical data. familiarity of setti and 
the already mentio. | AE 


expressin i 
digits ed р: Б numerical data in wi 


problems, 


BECHTOLD STUDY 


The hypot i 
isis els hesis that instruction in handling prob- 


ining superfluous data (in both i i 

Dm nau body Problems) had m au un 
m- soly; ity was rejected at 

€. The results with re- 


» intelligence E: 
O Significant differences ар ет 


sex апа intelligence. The data indicated, however, 
that younger students seemed to benefit more from 
instruction than older students. 


RELATION OF THE POST AND BECHTOLD 
STUDIES TO CONCEPT ATTAINMENT 


It is clear that if there is such a thing as a con- 
cept of problem solving, then it is a very complex 
concept. Moreover, success with problem solving 
is dependent upon mediating concepts that in them- 
selves are complex. The studies of Post and Bech- 
told, for example, underscore the importance of in- 
Struction, of exposure of pupils to many different 
sorts of problems or learning experiences in order 
that they have opportunities to attain necessary sub- 
sidiary concepts. 


The fact that familiarity of setting and superflu- 
ous data were significant factors in Post's study 
seems to indicate that pupils may have had no pre- 
vious encounters with such problems. Hence, they 
would have had no opportunity to form a concept of 
such problems, that is, to categorize them and de- 
velop strategies to use in their solution. Since mul- 
tiplication and division in problems turned out to be 
significantly more difficult to understand than addi- 
tion and subtraction, one infers that pupils do not 
conceptualize the former as well as the latter. It 
may be that the methods of teaching used with mul- 
tiplication and division failed to develop firm con- 
cepts. Asa result, pupils were unsure in their cat- 


egorizations when these operations were present in 
problems. 


Bechtold established that instruction in how to 
handle problems containing superfluous data led to 
greater success with such problems than that of a 
control group - at least for ninth-grade algebra stu- 
dents. A teacher is prone to extrapolate from this 
evidence and conclude that such instruction, appro- 
priately designed, is good for almost all age levels. 
Although the study does not represent conclusive 
evidence, still it is encouraging in that Statistically 
significant differences resulted. So often an exper- 
iment that compares two procedures in instruction 
ends with one group doing about as wellas the other 
on tests; there are no statistically significant differ- 
ences in the two sets of results. The unexpected 
significant differences in Bechtold’s study leads one 
to go beyond the statistics and say that experience 
with the analysis of problems with superfluous data 
significantly affects outcomes. 


From the definition of a concept given at the be- 
ginning of this paper, one sees that a person does 
not form a concept just through maturation. There 
must be experiences in his environment that make 
his attempts to understand the environment too com- 
plex unless he forms classes – unless he conceptu- 
alizes. This is Particularly true of mathematics, 
one of man’s sophisticated inventions. Perhaps a 
child develops “naturally” some concept of count- 


ing, even inventing his own name si 
in- 
stances. But as there are en Margery 
countered 
larger groups, the effort t ps 


oremember n 
names becomes too much, eed 
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that intensive exposure to a variety of experiences 

must leap-frog the historical pattern of development 
in mathematics. These learning situations cannot 

be left to chance; they must be provided; and they 

must be planned carefully. Experimental evidence, 
both in the field of psychology - learning theory — 
and mathematical education, furnishes guidelines for 
such planning. In turn, experimental evidence af- 
firms or denies that the planning is effective. The 

Post and Bechtold studies are examples of this two- 
pronged approach to a mathematics curriculum. 


SUMMARY 


So it is that human beings tend to deal with 
classes of things instead of individuals in order to 
make some sense out of their environment. By form- 
ing such classes cognitive strain is reduced as well 
as the burden on memory. To form these classes, 
or categories, or sets, a person looks for cues, or, 
if you like, for characteristics that serve to distin- 
guish things eligible for membership in the set from 
those that are not eligible. The point is that these 
categorizations are inventions, and this is particu- 
larly true in a complex body of knowledge like math- 
ematics. 


To solve a problem in mathematics, subsidiary 
concepts serve as mediators. Although initial focus 
and type of strategy have important roles, an indi- 
vidual will have little success in problem solving un- 
less he has a firm grasp of the mediating concepts. 
Mathematical education speaks in terms of a student 
mastering a concept so that it is a part of himself; 
a psychologist would speak in terms of a student in- 
ternalizing a concept. No matter what language one 
uses, it is certain that some of the failures in math- 
ematics instruction are due to an instructor assum- 
ing his students have understood a mathematical con- 
cept at a high level of operational thinking when in 
reality they have a much lower level of mastery. 
They cannot apply the concept in an unfamiliar set- 
ting. As experimental evidence accumulates from 
investigations of how people learn and from studies 
in a curriculum context, mathematical educators 
are on the one hand becoming more cautious and on 
the other more daring. Applying the evidence, they 
have changed the content of the curriculum and re- 
duced the amount of pure drudgery that use а to be 
the lot of every mathematics student. Assessing re- 
cent evidence, they have some doubts whether all 
Students master the early introduction of concepts. 


Already said is that people categorize the things 
in their environment. A categorization might result 
in an initial focus on a problem. Then certain hy- 
potheses are formed that are related to this initial 
focus, Decisions on these hypotheses serve to val- 
idate the hypotheses. 

The TE ence of decisions made [T the per- 

Son en route to attaining the concept which 

may be a problem solution or a pattern. ene 

may be regarded as a strategy embodying 

certain objectives. These objectives may be 

various in kind but in general one may distin- 
guish three kinds of objectives: a. to maxi- 
mize the information gained from each deci- 
sion and test of an instance; b. to keep the 
cognitive strain involved in the task within 


manageable or appropriate limits and certainly 
within the limits imposed by one’s cognitive ca- 
pacity; and c. to regulate the risk of failing to 
attain the concept within a specifiable time or 
energy limit... (Bruner, et al., 1956, p. 234). 


In the examples presented of problem solving or 
of attaining a concept in mathematics, the idea of 
initial focus and flexibility with respect to one’s fo- 
cus is of great importance. After isolating a key 
idea in a problem, a student has a choice of strate- 
gies – simultaneous scanning, successive scanning, 
conservative focusing, or focusgambling. A skilled 
problem-solver probably uses a combination of si- 
multaneous scanning and conservative focusing. A 
beginner is likely to use successive scanning. If 
time harasses a problem-solver, there is nothing to 
do but turn to focus gambling as a strategy. The sit- 
uation and the individual (his own experiences, bat- 
tery of concepts, and psychological make-up) deter- 
mine which strategy will be used. 


The investigations of Dienes (1963) at Harvardas 
well as that of Dienes and Jeeves (1965) in Australia 
lean heavily on the work of Bruner and Piaget. How- 
ever, their work goes beyond that of their predeces- 
sors, making use of new research results and sug- 
gesting refinements of the theory of learning upon 
which Bruner based his work. An idea that Dienes 
and Jeeves, separately and together, experimented 
with is the development through ‘‘mathe matical 
games” of the concept of isomorphism of structures. 
Involved here is the recognition of patterns and no- 
ticing the similarity between two sets of data and 
the structure that can be applied to both. 


Suppes (1965) has done a great deal of work with 
the major emphasis on learning models for relative- 
ly non-complex mathematical concepts. He charac- 
terizes his work by saying that it is based upon a 
stimulus-sampling theory. An interesting assertion 
is that the hypothesis-strategy language of Bruner 
and his associates can be related to stimulus- sam- 
pling terms: ‘‘... a strategy ... corresponds pre- 
cisely to a state of conditioning and a hypothesis to 
the conditioned stimulus sampled on a given trial 
..."' (Suppes, 1965, p. 65). 


Some psychologists doubt the value of a mathe- 
matical psychology approach to models for human 
learning, but there is no disagreement that much 
work needs to be done with complex concepts in or- 
der to learn more about individually different ap- 
proaches to attaining mathematical concepts, Mean- 
while, the mathematical educator makes use of 
available knowledge, applies it as best he can to the 
mathematics program, and gathers experimental 
evidence to see the effect. It is impossible to wait 
until definitive psychological research on learning 
has been done, for the curriculum exists and must 
be modified to fit twentieth century activities. One 
implication is clear: new psychological evidence 
should be brought to bear on thecurriculum as 
quickly as possible. Itisinthe latter direction that 
mathematical education is turning. 


NOTE: Situations 3 and 4 are modeled after similar 
materials that appeared in the Nuffield Foundation 
Mathematics Teachin| Project Bulletin No. 1, Nov- 
ember 1964, pp. 4-7. 


(000000 et” 
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umm PART 111 


mamme Instructional Materials and Media 


THE MEDIUM of instruction can be considered as the interface between 
the learner and the subject matter being learned. It necessarily affects the rate of 
learning and imposes limitations on the scope and sequence of the instructional sub- 
ject matter. Instruction by television has been criticized as impersonal and encourag- 
ing passivity. Yet, according to Wilbur Schramm in "Instructional Television Around 
the World" instructional television can, when properly used, provide for active 
learning experiences. In a series of case studies, Schramm reports on the use of 
television in widely varying sections of the world—from the affluent suburb to 
the developing nation. Properties and attributes of the media are discussed with 
reference to the school system, the teacher, and the student. Television is exam- 
ined as а potent mechanism of change. Schramm concludes that this medium 
Works best when it is integrated into an instructional system which includes special- 
ized materials, activities, and teaching techniques. But the key to the system is 
a local Supervisor through whom students can interact. with one another. 

Research dealing with the process of instruction is examined by Arthur 
Lumsdaine who points out that problems associated with instruction e fact 
inseparable from problems dealing with learning. The close relationship йи i 
"esearch on instructional media and the more general field of basic psyc ology 
d learning must be understood by persons purporting to do erp ы bas 
tional media. Among the general areas identified for research on the ге Ма 
Struction is the ‘far underdeveloped research field of or piety MET k 
tive attempts to improve on the basis for determining ME dis кё 
lumsdaine argues that determining the effects of present i 


Р і + for these 
Precede ihe "more elegant question of what specific variables accoun 


RES ; i a prerequi- 
Effects.” He takes the position that effective instructional media are a p q 


i is impli rams of devel- 
dle to meaningful basic research on these media, This implies programs ei Cave 
°PMent before research and suggests a basis for priorities in "do e then have an 

w 
ment. "Only when something effective has been developed, do 


adequate tool with which to conduct useful research. 


The question of cost efficiency which is usually not considered by 
educators is explored. Such efficiency frequently implies a measure of the level of 
proficiency achieved for a given effort with, as Lumsdaine suggests an approach 
in which the time required to achieve a given minimum level of proficiency is used 
as the dependent variable. Such a proposal has significant implications for the 
design of instructional objectives and instructional research as applied to selected 
subject areas, 

Lumsdaine concludes with a discussion of the standards or lack of 
standards used in the design and reporting of research studies dealing with instruc- 
tional media. 


Matching the instructional strategy to the capabilities of a learner re- 


mmarizes a general idiographic 
computer based instructional system called 


Instructional Television 


Around the World 


body of comparable and detailed data yet assembled 
on instruction: i 


; most of them in- 
Iwill review three of these 
cases in some slight detail, cite a few others, and 


then try to sum up what I think the studies Say about 


the requirements for using instructional television 
effectively. 


One point I shall not belabor is that instructional 
television, used well, clearly does work. It works 
over a wide spectrum of tasks and a variety of situ- 
ations. It works in developing countries and coun- 
tries that are highly industrialized. It works some- 
times very well and sometimes ordinarily. To illus- 
trate what these cases Say about how it works and 
under what conditions it appears to work well, I will 
enumerate three instances. 


CASE 1: SAMOA 


The story of instructional television in Samoa 
really begins with the appointment of atough, deter- 
mined, little governor, Н. Rex Lee, in 1961, but the 
Whole story is 60 years older than that, and deals 
mostly with American neglect of its wards on the 

amoa islands, The school system was only one 
Symptom of that neglect, but it was a vivid one. 


The Un 
Е Samo: 
Оп the m, 


ited States had committed itself to offer- 
ans an education equal in every way to that 
аш апа. Yet the Samoan schools were in 


im 


ccm Wilbur Schramm 


Stanford University 
Stanford, California 


one-room, open fale huts, where two classes com- 
peted with each other, fortissimo. The instruction 
itself was the most traditional rote learning. I have 
been in these village schools and have heard the kind 
of choral chanting that one becomes accustomed to 

in primitive schools throughout the world. The stu- 
dents repeat back — chant back ~ what the teacher 

gives them; indeed, this is the only kind of teaching 
possible when the teacher is untrained, little edu- 
cated, and not at all confident of his 
ject matter. He doesn’t dare encou 
heir own intel- 


lectual excursions, The teacher says, ‘‘Class, what 
am I doing?" 


Like one, the class chants back, **You - are- 
standing - on - the - mat,” 

“Class, what do I have in my hand?" 
questions. 

'*'You - have - a - piece - of - chalk,’ 
replies. 

Once I rose, smiled as disarmingly as possible 
at the teacher, and said ‘‘Class what am I doing?” 

“You - are - standing - on - the - mat," they re- 
sponded promptly, although I was far away from the 
mat. 

I picked up a book. This puzzled them; they could 
see it wasn’t chalk. ‘What do I have in my hand?’ 

One boy ventured: ‘‘You - have - а - piece - of- 
chalk. ” 

They were quite unable to generalize much upon 
the responses they had learned in their educational 
catechism. 


the teacher 


' the class 


The results of this kind of thing were predictable, 
The schools were supposed to be the equals of main- 
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land schools, but children graduated from the 
twelfth grade testing on the average from f if th to 
ninth grade in different subjects. The teaching was 
supposed to be in English, but little of it was, and 
many of the teachers could not speak the language 
understandably. The content was derivative from 
mainland texts, and so the children learned their 
reading vocabularies from accounts of commuter 
trains, milk men, and snowballíights. Isaw one 
elementary- school class being drilled to remember 
that ‘Тһе ‘Mona Lisa’ is in Milan, ’’ although the sig- 
nificance of that knowledge to a Samoan child was 
not immediately obvious. Andover everything hung 
the pall of traditional rote instruction. 


Now, how do you transform a traditional school 
system like that, in a few years, into a modern 
problem-solving, inquiring school system? How do 
you live up to the promise and plan to teach in Eng- 
lish, with so few English models? That is the prob- 
lem Governor Lee faced. Iam begging the question 
of whether the level should have been raised ina 
few years, rather than a century; whether it was im- 
portant to have English well taught and well used ; 
whether an excellent 12-grade school system was 
what Samoa most needed. But given that policy goal, 
how would you proceed? 


What were Governor Lee’s alter natives? He 
could replace all three hundred Samoan teachers 
by qualified teachers from the mainland, if he could 
recruit them. But this would have been a politically, 
psychologically, and ethically impossible solution. 
He could bring in perhaps one hundred teachers 
from the mainland. This would displace less than 
one-third of the native teachers and stirup less 
trouble, but would run head on against one of the 
oldest Samoan norms - that allgoforward together, 
He could institute a greatly increased teacher pro- 
gram. But this would require sending prospective 
Samoan teachers to the mainland for some years – 
lest they merely perpetuate the quality of the old 
School system — and its effect would not be felt for 
perhaps 15 Years. Or he could use te levision to 
pert teaching over many 
his is what he decided to do. 


Share a small amount of ex 
Classrooms. And t 


ch ateaching- 
Nowhere in 


Governor Lee persuaded Congress that our neg- 
lect of Samoan education should be repaired. А re- 
markable installation was built at Pago Pago, one of 
the finest instructional television installations in the 
world, with six open-circuit channels, ten videotape 
recorders, transmitters on a mountain top, and tel- 
evision teachers, engineers, and producers г e= 
cruited from the States. Classes began to move in- 
to the new school buildings in 1964, and television 
began to carry the burden of teaching beginning 
in the autumn of that year. 


How is it working? What evidence is there that 
television can raise a primitive toa modern system- 
Beeby’s (1966) Class I and II schools to his Class 
Ш and IV schools — in a short time? 


The evidence isn't all in, yet, but the signs are 
encouraging. The Samoan elementary schools are 
working very well, indeed. Thechildrenare speak- 
ing much more and much better English. The tele- 
vision classes are live and active; the pupils talk to 
the teacher and answer her questions just as though 
She were in the room. There is much less rote 
practice, Children are asking questions, Some 
Schools are making barometers and undertaking 
other special projects. I sawone sixth-grade teach- 
er bring in a lizard for the class to study. They 
talked about the little beast, what he ate, who ate 
him, and worked around to the balance of nature, 
This could not possibly have happened only a few 
years ago. Ican’t say that the high schools - which 
are using, mistakenly or not, very large classes - 
are working as well as the elementary. But on the 
whole, it is a very encouraging and hopeful experi- 
ment, one that is worth watching. 


Pause briefly, ase, 
to reflect on what seem to be the basic elements of 
the Samoan case: a problem, large, urgent, im- 
portant enough so that many groups can r ally to 
solve it; adequate Support, administrative and fi- 
nancial; and the introduction of television, not as a 
separate element, but as a part of a teaching-learn- 


ing system integrating all the teachers and all the 
classroom activity. 


before we review the next c 


CASE 2: HAGERSTOWN 
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subjects by means of television, at a lower cost,or 
at least no higher cost, than by hiring the necessary 
specialized teachers. In the second place, they 
wondered whether in the coming years of expanding 
enrollments there would not be some advantage in 
designing schools, not with many rooms about the 
same size, but rather with a central large room or 
rooms for viewing televised classes, and a number 
of small rooms around the perimeter of the building. 
And, finally, financial help was available — $1 mil- 
ion from the Ford Foundation and over $300, 000 in 
equipment from manufacturers. 


This was in 1956. Hagerstown decided, as did 
Samoa later, that it needed the equivalent of six 
channels to teach the core of an entire curriculum. 
Because open-circuit channels were not available in 
that number at any one point in the United States, it 
was decided to go closed-circuit. Coaxial cables 
were leased from the telephone company, and ex- 
tended gradually over the next few years to every 
school in Washington County. For more than a half 
dozen years, now, every student in the system has 
received the core of his curriculum from television. 


In Hagerstown, as in Samoa, a studio teacher 
and a classroom teacher operate as a team to teach 
content previously decided upon. The studio teach- 
ers are chosen from the regular teaching corps of 
Hagerstown, and are changed frequently. Because 
all the teachers in the system are qualified and well 
trained, there has been less difficulty, inthe Hagers- 
town than in the Samoa experiment, in getting the 
classroom teacher to play his new role ex pertly. 
But two elements are generally credited with the 
smooth transition in Hagerstown fromthe older plan 
to the television teaching team. One of these is the 
firm support given the project by Superintendent 
William Brish. Ina sense he played the role in 
Hagerstown that Governor Lee played in Samoa. The 
second is the insistence throughout on involving as 
many teachers as possible and thus reducing the 
potential threat of the new method. Only about 5 
percent of all the Hagerstown television teaching is 
taped, and very little of it is brought infromoutside. 
Thus teaching on television is treated as an ordinary 
function of team teaching, not specially rewarded, 
and passed around among promising candidates. 


A considerable amount of research has been done 
at Hagerstown, and some of the results of introduc- 
ing television have been spectacular. Remember, 
now, that this is not a deprived school system, nor 
was it in 1956. It is a rather average American 
system. But in Hagerstown junior high schools dur- 
ing the first four years of television the average 
performance of students on a standardized test of 
arithmetical concepts rose from the thirty-first to 
the eighty-fourth percentile, measured against na- 
tional norms, and on a standardized test of problem 
Solving it rose from the thirty-third tothe sixty- 
eighth percentile. Inother words, before television, 
two-thirds of the junior-high-school pu pi ls ofthe 
Country scored above Hagerstown children on these 
arithmetic tests; after television, two-thirds scored 
below Hagerstown. In the fifth grade, during the 
first year of television teaching, the average Hagers- 
town pupil gained 1.9 years, rather than the one 
year represented by national norms. In the year 
before television was introduced, every one of the 


grades three through six averaged below national 
norms in arithmetic. Within 2 years, each grade 
averaged above the national norms. Wehave spoken 
only of arithmetic. Not all results were as good as 
these, but the gains were by no means limited to 
arithmetic. Perhaps the most significant verdict 
was turned in by the school board of Washington 
County who, in 1960 when the Ford grantcame toan 
end, studied the results, gave all teachers and ad- 
ministrators a chance to express their opinions 
anonymously, and then voted without a dissenting 
voice to take over the approximately $600,000 annu- 
al cost of the television in their own budget. 


CASE 3: NIGER 


Both Hagerstown and Samoa, one might say, prof- 
ited by American know-how in television. Suppose 
now we turn to a country that has had little or no 
help from the United States in television. Niger has 
only sixty-six fully qualified teachers - meaning 
teachers who are secondary school graduates and 
have had teacher training. There are some hundreds 
more who have had the equivalent of junior high 
School education, and still more who have had only 
5 or 6 years in school. But there is still no hope 
for adding to the resource in highly educated and 
trained teachers until the country's higher priority 
needs for educated people have been met. This cre- 
ates an extremely difficult situation because Niger, 
which has had only between 5 and 10 percent of its 
school-age children in the classroom, is now trying 
to double and soon to triple its school population, 
and especially to strengthen its secondary schools. 
The problem was, then, how to find the necessary 
number of skilled teachers. 


Acting with the advice and helpof the French 
government, Niger developed a plan for using expert 
teachers in the studio and monitors inthe classroom. 
Unlike Samoa and Hagerstown, which began with all 
the elementary and secondary grades at the same 
time, Niger began with the first grade. It began 
very cautiously, in fact, with only two Schools, and 
with closed-circuit television. By the timethe coun- 
try's new open-circuit television station went on the 
air, the first-grade experiment had shaken down 
and the television was expanded to the second grade. 
The plan is to add one grade a year. 


Like Samoa and Hagerstown, Niger has found it 
works best to create in effect a teaching team in the 
studio and the classroom, and like Sa moa it has 
found that the weakest link in the team is the class- 
room teacher when that individual is not adequately 
trained. Using elementary- school-trained monitors 
in the classroom, the supervisors of the Niger pro- 
ject have tried short in- service training cour Se 5 
and have decided more intensive Specialtrainin е 
is needed. One of the ways Samoa tried to spline 
the problem of helping the relatively untrained class- 
room teacher to learn his new role was by puttin, in- 
to every school an American principal, who wo bed 
with his teachers and served really as a super ja 
They also brought all the teachers to a s deas T 
summer workshop each year, and conducted coe 
in-service training by television. No easy ile Y 
сева am anywhere to solve this particular EQUIS. 

ut it has reappeared again and again in developing | 
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countries where instructional television has been 
introduced. 


One of the distinctive features of the Niger exper- 
iment has been the continuing presence of a three- 
man research team supplied by France — a psychol- 
ogist, a sociologist, and an ethnologist. These peo- 
ple are studying very closely the effect of television 
teaching on African children who have never before 
seen television, and, most of them, have never be- 
fore spoken French, which is the language of instruc- 
tion. Their psychological and cultural reports are 
not yet available, but they have measured the learn- 
ing results of the first year. As might be expected, 
the results were better in some subjects than others, 
and somewhat better for children who were a year 
Older than the others. Butasa whole, the televi- 
Sion classes averaged well above the average per- 
formance of non-television classes. We musts ay 
of Niger, as of Samoa, that all the evidence has not 
come in, but the outlook is highly encouraging. 


SOME NOTES ON OTHER CASES 


We will deal with several additional cases ina 
more brief manner, 
You have noticed that in all three of the cases I 
have mentioned, generous financial help was avail- 
able, and in the case of the twodeveloping countries 
expert counsel was also freely offered. What would 


happen now if a developing country tried to go it on 
its own? 
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mostly for out-of-school use. There have been a 
number of other projects of that kind, aimed at ex- 
tending educational opportunities beyond the school — 
Íor instance, Telescuola which provides secondary 
education for Italian children who do not have sec- 
ondary schools in their areas; and the Chicago Jun- 
ior College which for nearly 10 years now has of- 
fered a complete junior college curriculum by tele- 
vision, with the home television students doing con- 
Sistently better than classroom students on the same 
examinations. Television has been used tohelp 
teach literacy — in places like the Ivory Coast where 
it has helped to train native workers to read, write, 
and speak French, and do simple arithmetic sothat 
they could be made middle-level supervisors; and 
in Italy where about 15, 000 adults annually register 
to study reading, writing, and arithmetic, with the 
aid of the famous program series, ‘‘Never Too 
Late.” The trend of the evidence in literacy exper- 
iments, though, is that, whereas television can as- 
sist greatly by providing visual material, by moti- 
vating a group, and bringing a certain amount of ex- 
pertness, the rea] key to what happens lies in the 
ability of the group supervisor. A Straw in the wind 
is that Italy, which at first used only monitors in 
its viewing groups, now uses certificated teachers - 


usually working outside school hours — for that pur- 
pose. 


One more note: from almost everywhere instruc- 
tional television 
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CONDITIONS OF SUCCESS 


(1) A problem. 


After twenty-three case studies, I devoutly hope 
that the next time someone comes to me for advice 
on instructional television, he comes witha problem 
and not with a piece of technology. This is why 
some of the more spectacular examples of use of 
television for teaching have come in developing 
areas like Samoa and Niger, Colombia and Peru. In 
those cases the problems have been so challenging 
that it has been possible to think boldly of new ways 
of solving them, and to get sufficient support behind 
the effort. Ithink it is fair to say that most Ameri- 
can school systems have not felt quite that order of 
need for television. It is likewise true that most 
American schools which have tried television have 
been the schools that needed it least. Perhaps not 
until we turn the power of instructionaltelevision on 
our deprived schools (which are to be found on the 
mainland as well as in Samoa and the Trust Terri- 
tories), or to meet the likely expansion of schooling 
from ages 3 to 20, with thirteenth and fourteenth 
grades appearing in every city of over, say, 15,000 
people, will weourselves be challenged to use 
the medium to its full capacity. 


(2) A teaching-learning system. 


Instructional television hasn't worked well when 
it has been used like films — at the option and at the 
time of particular need felt by the teacher. Perhaps 
Some day when we have enough inexpensive video- 
tape recorders in the schools it may be feasible to 
use it that way. But the way it has worked most ef- 
fectively, whether in developing or developed coun- 
tries, is when it is integrated into a teaching-learn- 
ing system — not when it is used alone, or when it is 
used as an optional visual aid. This requires the 
teacher in the classroom to learn a new role, and to 
many teachers at first this role may be threatening 
or uncomfortable. This has been the sourceof most 
teacher resistance, where it has occurred. But ex- 
perience indicates that the role need be neither 
threatening nor unpleasant; in fact, it may be more 
satisfying than the role of being the only teacher in 
the classroom and in complete command of every- 
thing that happens there. Andonce the transition 
period has been passed, both teacher and student at- 
titudes toward the new way of teaching improve stead- 
ily with experience with it. The experience of work- 
ing in a cooperative team of studio teacher, class- 
room teacher, materials expert, and othersas need- 
ed, can be a most satisfying one, once the roles are 
learned. 


I would like to digress and suggest that one of the 
old stereotypes about television teaching – that it is 
а passive experience for the pupils – isdestroyed in 
every place where instructional television is being 
used effectively, The classes are extremely active. 
The pupils talk to the teacher on the tube, practice 
as directed, answer the teacher's questions, pay as 
close attention, and, in elementary schools at least, 
Seem to enjoy the experience at least as much as 
with face-to-face instruction. The television-teach- 
ing-team is not an encouragement to passivity, but 
rather an opportunity to stimulate learning activity. 


(3) Support. 


Any innovation needs support. Instructional tele- 
vision, which is expensive and somewhat disruptive 
to old roles, and which requires obedience to central 
Schedules, needs it especially. Governor Lee in 
Samoa, who was strong and determined enough to 
meet the resistance he encountered from certain 
mainland teachers who did not believe in television; 
Superintendent Brish who was firm without being 
rough; the President and other chief officials of Ni- 
ger who quieted some incipient uneasiness by them- 
selves coming to visit the experiment — this kind of 
support is priceless in the infant years of instruc- 
tionaltelevision. And beyond that, thefinanc ial sup- 
port necessary to have an adequate technical service 
and an adequate nucleus of skilled personnel ,is 
a sine qua non. The magnificent technical equip- 
ment of Samoa has made a difficult task much easier. 
On the other hand, I could tell you about small st a- 
tions where the power is forever failing, where re- 
ceiving sets are not maintained, where teachers are 
struggling with little help, little feedback, scant ex- 
perience. I do not have to tell you that there is little 
chance of using ITV effectively under conditions like 
those just cited. 


(4) Size. 


There is a certain critical mass beyond which in- 
structional television seems to make an important 
impact and works most efficiently. Part of this, of 
course, comes from the economies of scale which 
are made possible by larger operations. Colombia, 
for example, can deliver televisionfor about 5 cents 
per pupil hour, but it is serving a quarter of a mil- 
lion pupils. Some unit costs, where the system is 
notably under-used, run as high as $2 per pupilhour. 
We find that in most systems, instructional televi- 
Sion is under-used rather than over-used. Spectac- 
ular differences in unit costs can be achieved even 
by balancing up the investment in programs against 
the investment in receiving equipment. This is 
something we cannot develop in this paper, but you 
can find details in the case studies, Beyond the fi- 
nancial effect, though, there is a psychological im- 
pact that comes with size. It is impossible not to 
take the Samoa operation Seriously. It is impossible 
not to take Hagerstown seriously. Theteach ers 
the students, the communities know it. Observers. 
have reported from several places that the very fact 
of the existence of a considerable ITV operation has 
made all developmental change easier in that area, 


(5) Planning. 


Let us merely say that most countries 
systems have bumbled into instructional prio 
Many of them have wakened and found it under the 
Christmas tree — the equipment being intended take 
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cation. Many countries that have launched into + i 
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learning system like television, for a school System 
or a developing nation to sit back quietly for a while 
and consider goals and means and desirable direc- 
tions of change. 


There is no magic about instructional television. 
It is merely a pipeline into the classroom ~ a way 
to share and redistribute teaching resources. And 
yet it puts a wand of potential magic into the hands 
of any school system that has a real need to use it 
and the willingness to learn to use it well. 


FOOTNOTE 


1. The countries (or other political units) were Al- 
giers, Australia, Colombia, Ghana, Honduras, 
India, Italy, Ivory Coast, Japan, New Zealand, 
Niger, Nigeria, Peru, Samoa, Thailand, Togo, 


United States, UNRWA. The case studies 
have now been published under the title, New 
educational media in action: case studies for 
planners. Paris: UNESCO and International 
Institute for Educational Planning, 1967, 3 
vols. An interpretive volume has been pub- 
lished with them, Schramm, W., Coombs А 
Р. H., Kahnert, F., and Lyle, J. The rew 
media: memo to educational planners. Paris: 
UNESCO and International Institute for Educa- 
tional Planning, 1967. 
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= Instructional Research: Some 


Aspects of Its Status, 


I WOULD LIKE to structure my pa- 
per in terms of the following main topics: (l) in- 
Structional- media research as related to research 
on learning and instruction more broadly; (2) 
classes of problems which are involved in the 
assessment of instructional effects, both at the ap- 
plied level and at the ‘‘basic research" level; 
and (3) some considerations regarding the priori- 
ties of what it is most important to do in the in- 
structional-research field at the present time. In 
relation to the question of priorities, I will also 
review the status of alleged ‘‘findings’’ of hope- 
fully relevant basic research; some serious defi- 
ciencies of such ‘‘findings’’; and some notions con- 
cerning requirements for improving the state of 
research practice. 


PUTTING INSTRUCTIONAL-MEDIA 
RESEARCH IN CONTEXT 


The first point I should like to make isone which 
people working in instructional- media research 
have realized in varying degrees for a long period 
of time. This is that the important problems in in- 
Structional- media research are actually broader 
problems of research on the effects of the manage- 
ment of learning and hence basically research on 
learning. Most of the laboratory researchon learn- 
ing has at least potential, even if unrealized, rele- 
vance to the design of instructional materials and 
procedures. However, much of it does not seem 
very useful when one tries to apply it to practical 
problems of instruction - as those who teach 
courses in the psychology of learning often realize 
when they attempt to apply their own principles. 


Arthur A. Lumsdaine 


University of Washington 
Seattle, Washington 


Most of the important problems for research on 
basic variables in instruction--i.e., stimulus and 
response variables influencing the effectiveness 
with which instruction produces learning--have re- 
ceived some attention in the field of instructional- 
media research. In my opinion, in fact, more in- 
cisive research on instructional variables has, thus 
far, been done within the instructional- media field 
than in any other context. Ithink that this state- 
ment is not just a reflection of a parochial bias on 
my part; rather, Ithink it reflects the fact that the 
encapsulation of instructional procedures in specific 
media which are mechanically reproducible 
(like films and self- instructional programs) gives a 
degree of control over the variables involved in pre- 
sentation, exposition, instruction, and persuasion 
that is very hard to achieve in other forms of re- 
search. Where a particular experimental treat- 
ment can be carefully and deliberately implemented 
in advance, as in a film in the cutting room, there 
is much less dilution from the concept, or variable 
or procedure presumably being implemented, to its 
actual implementation in the process of instruction 
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of the Skinnerians don't like to think of it that way). 
“Instructional media, ” in turn, is part of the still 
more general field of instruction, which isa special 
aspect of the more general field of human learning. 
At each level it seems desirable to goback to basic 
learning principles where possible, asa source 

(but not the only source) of hypotheses or ideas for 

development and test. 


Not all of the good Suggestions for effective in- 
struction come as safe deductions from Hull’s pos- 
tulates or any other simple, single source. Such 
suggestions also stem partly from analysis of sub- 
ject matter--e, g-, in Gagné’s analysis of learning 
Sets--and partly they arise in other ways. For ex- 
ample, in the process of developing Specific instruc- 
tional products one learns а great deal about what 
variables seem to be important or worth looking in- 
to. One formulates hypotheses that a learning 
theorist, as such, would never think of when one 
Starts to attack Specific, concrete problems in edu- 
cation. A related point which has been made in 


View, for people who wish to do fruitful basic re- 


things that learners really have to learn. On the 
other hand, I also think that it is important, in try- 
ing to analyze the essentials of instructional- media 
(or other instructional) Situations, to have at least 
a good talking vocabulary with which to interact 

with people whose main concern isthe basic 
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field, in its most i 


closely to the more general field of research on in- 
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RESEARCH ON THE EFFECTS 
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in any given instance?" Third, ‘‘What accounts for 
them?" (This is where a great deal of research 
emphasis has been. ) Fourth, ‘‘How do you predict 
them?" Fifth, “Ноу do you improve them?" And 
sixth, ‘‘What is the cost (and ‘cost- effectiveness’) 
of improving them--how much is worth paying for 
а given amount of improvement?" 


WHAT EFFECTS SHOULD BE 


A number of people have pointed out that in 
evaluating instructional programs there are sever- 
al kinds of criteria used. In the report which was 
published by the Joint Committee on Programmed 
Instruction and Teaching Machines (1966) three 
classes of criteria were distinguished: first, 
appropriateness of various possible effects; second, 
the effectiveness with which sought-for outcomes 
were brought about; and third, the feasibility of using 
the materials, including cost. Considering just the 
first two, it is quite evident that one can do a ve ry 
effective job of teaching the wrong thing. (It could 
even be held that doing an ineffective job of teach- 


to be taught. Very little effort has been put forth 
on this as compared with approaches on how to teach 
a given thing more effectively. This applies all the 
way from kindergarten to graduate work, 


ences 
of the obvious fact that a good film will a 


lways 
beat a poor lecture and vice versa, and of the fact 
that such terms as “conventional instruction” have 


been so poorly defined as to be almost completely 
ungeneralizable, 
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which is too limited to be definitive) a groupof ‘‘ex- 
perts'' (students who had recently been through a 
course in the ‘‘proper’’ way to construct instruc- 
tional programs) were asked to rate seven pro- 
grams. These seven programs all dealt with the 
same subject matter and, without the knowledge of 
the judges, their actual effectiveness in teaching 
specified kinds of outcomes had been previously 
determined by trying each of the programs on seven 
randomly selected groups of students. When rated 
effectiveness in that set of seven programs was 
correlated with measured effectiveness (as meas- 
ured by what students actually learned), Rothkopf 
concluded that the instructional effects were indeed 
*predictable': the correlation was minus .75. All 
that need be done was to get judges to rate them 
and then do exactly the opposite. Such data support 
the contention that the effects of instruction are not 
really very well predictable on the basis merely of 
inspecting the features of the instructional materi- 
als and deciding whether they conform to the im- 
plications of a ‘‘science of learning" or to the ac- 
cumulated wisdom of teachers. This means that 
either we are applying the wrong findings or we 
have a great deal more to find out about the effec- 
tive variables in instruction. This in turn takes us 
back to Question Two, What are the effects of in- 
struction? 


Icontend that we might well set for ourselves 
the perhaps modest objective of finding out what we 
are actually doing in much on-going instruction. I 
think we at least need to know what effects we are 
producing before putting such exclusive concentra- 
tion of effort into the more elegant question of what 
specific variables account for these effects. In many 
of the studies of instructional effects, one problem 
has been the lack of significant differences, report- 
ed with dismal uniformity. This lack would have 
been predictable if the investigators had bothered to 
runan initial test of any one treatment of the instruc- 
tional materials (preferably the best), using avail- 
able criterion measuresof output. Often this would 
have shown them that, іп fact, the whole program 
was teaching at such a low level of efficiency that 
the likelihood of revealing any differences among the 
treatments was indeed very poor. All one has todo 
is to read the instructional research literature to 
find many instances of this kind, and I must even 
admit that this has been true of some of the studies 
which I have been responsible for sponsoring. It is 
avery simple truth, but one which has eluded inves- 
tigators for a long time, that if we are going to in- 
vestigate factors that make instruction effective we 
hàve to get the instruction effective in the first 
place and then find out something about how itgot 
that way, instead of working hard at manipulating 
variables in a piece of instruction which we later 
find out is operating at a low level of efficiency. 
One of the implications of this is that the classical 
sequence of research and (then) development often 
must be reversed, In other words, one must do the 
development fir st; only when something effective 
has been developed, do we then have an adequate 
tool with which to conduct useful research. This is 
an oversimplification, of course, but it remains a 
crucial and an all-too-often ignored consideration 
in the instructional research currently being done 
(and continuing to be supported). 


Regarding Questions Four and Five (How to pre- 
dict the effects of instruction and how to improve 
them), it may be noted that both imply important 
kinds of purposes. However, it can be argued that 
prediction is of no use in itself unless it can ulti- 
mately be used to bring about better instruction. 
One way to do this is through better selection of 
existing materials (as well as making better ones). 


The conduct of adequate empirical evaluation 
Studies for this purpose, which would require the 
detailed testing of every piece of marketed instruc- 
tional material, is an extremely time- consuming 
business. It would obviously be a great advance if 
опе could reliably predict which materials would be 
more effective than others, as a basis for selecting 
Írom available materials within a given set of in- 
Structional purposes. In any case, the ultimate ap- 
plied purpose of fundamental knowledge about the 
influence of instructional variables on learning out- 
comes is to be able to know how to improve the in- 
Structional product. 


A vexing problem of instructional research for 
this purpose concerns the extent to which the inde- 
pendent variables named by the investigator are 
idiosyncratic to him. For example, in experiments 
on ‘‘student participation” or ‘‘varied repetition, ” 
(or you name it), the independent variable is seldom 
defined in terms sufficiently operational to provide 
reasonable assurance that someone else would be 
doing essentially the same thing if he said he were 
studying the nominally ‘‘same variable.’’ That is, 
for most of the pedagogical variables derived from 
either common teaching practice or learning theory, 
the specific way in which the ‘‘variable’’ is imple- 
mented can turn out to be a great deal more impor- 
tant than what variable it nominally is. We, there- 
fore, have findings in which the ‘‘same variable” 
(e.g. , overtness of response) turns out to have one 
effect when explored by one investigator and another 
effect when explored by another investigator. The 
conclusion would seem to be that, pending such 
time as we are better able to define what our in- 
Structional variables mean in reproducible, opera- 
tional terms, we had better assume the burden of 
demonstrating explicitly the generality of our find- 
ings. The deduction from this is that in an instruc- 
tional experiment in which one manipulates a named 
variable or procedure, the critical number of ‘‘de- 
grees of freedom” on which generalization forms 
the experiment depends is a function of the number 
of experimenters (or programmers), rather than 
the number of subjects (learners). Put another way: 
if we are really going to say that we have found a 
dependable rule or generalization about an instruc- 
tional procedure, we have got to show that we ca 
write a set of instructions which will allow another 
experimenter, programmer, or teacher to Create 
substantially similar effects to those We obtained 
I know of only one or two studies (still underway) ir 
which this is systematically being done. In na a 
these studies the number of instructional vehial E 
representing each treatment are Written by six o 
ferent programmers following a given inst 7 
as against six other programmers who are 
a different instruction. The number of 


є » 12, 5, or 6, dependin; 
kind of statistical test is used, but it t gei re 
not be a 


98 


function of the number of students or classes that 
are taught. 


This very important point can be summarized 
simply by saying that we have to be able to show, 
in a scientifically acceptable fashion, that someone 
else can manipulate and implementthe variable un- 
der consideration. At present, I know of no way to 
do this, for most variables at least, except by brute 
empirical demonstration. This is one of my main 
reasons for the quotes around the word “findings’’; 
for, of the experimental results on instruction, 
very few, if any, have been systematically repli- 
cated ina way that provides this kind of formal 
demonstration of their generalizability outside of 
the particular program or programmer responsible. 


To consider Question Six (concerning cost and 
cost effectiveness) requires that we first consider 
basic objectives. In terms of the kinds of outcomes 
we are trying to achieve, we have ultimately got to 
translate progress toward these objectives into 
Something that is cost accountable. Ithink the 
things that have the best chance of b 
accountable are things that are correl 
rather than being correlates of achie 
For this reason and other, I think a Strong case 
can be made for Skinner's contention that, in effect, 
we should use as our dependent variable the time 
to achieve a given minimum criterionof proficiency, 
rather than the levels of proficiency attained after 
a fixed number of hours of instruction as shown by 
the resulting distribution of Scores, 


ecoming cost 
ates of time, 
vement level, 


ter. There is simp 
to know and too much 
and part of the proble 
ly, going to be to det 
get “kept” і 


how much 
instruction 


get back up to proficiency, 
takes him t 


a person can 
rather than how long it 
О get there in the first place. 


A difficuli 
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ment test nor ms seems much more puzzling, and 


possibly involves the wrong kind of educational ob- 
jective anyhow. 


RESEARCH AND 
DEVELOPMENT PRIORITIES 


QUANTITATIVE PRODUCT-EVALUATION DATA 


I believe that a great deal more work is needed 
on the simple, modest question of assessing or 
measuring the effects of available and reproducible 
instructional programs; I think this perhaps un- 
glamorous area rates a ve ry much higher priority 
than it is getting. We need to know what we are 
accomplishing, and by and large we don’t know that. 
The programmed- instruction people have banged 
the drum for data accompanying programs, and we 
have even set up some standards so that we could 
have some hope that a sort of Gresham's Law 
wouldn't operate with bad data driving out good data. 
Yet at present I know of only a ver y few programs 
on which any reasonably adequate data have been 
published. Somewhat more than that number of pro- 
grams have something which purports to be data, 
but which are merely records of what a few students 
did as they wentthrough some version (not necessar- 
ily the published one) of the self- instructional pro- 
gram. For the present I would allocate a much larger 
share of research budget to support lowly, “dirty, © 
applied work on getting product specifications 
for available programs that potentially will have 
wide use, and at least relatively less of it to what 
are, all too often I think, futile attempts to do fun- 
damental research. (Iam well aware that some of 
my colleagues will r egard this as utter here Sy.) 


THE PROBLEM OF WHAT TO TEACH 


Another act of high research priorities is related 
to Question One, – What should the effects of in- 
Struction be? We havean enormous range of opinion 
on this. We also have an increasing amount of 
knowledge to be taught and yet no rational basis for 
Selecting what we should teach, not only to graduate 
Students in Specialized areas and to undergraduates 
who are acquiring a *'libera]'" education (presum- 
ably to prepare them in some fashion to deal with 
the problems of the world), but even, to a consider- 
able extent, to the elementary- school children. The 


Sget somewhat easier at 
levels, as th 


School stu- 
d scientific in- 
ry. 


In any case, I believe it would be 


try to do more extensive Survey rese 
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taught are ultimately transfer-of-training ques- 
tions. If a more sustained attempt were made to 
define and characterize the kinds of educational 
outcomes we really want, then we could try to veri- 
Íy some of the working assumptions which we seem 
to make as to what kinds of learning transfer to 
these outcomes. We would be then moving in the 
direction of an experimental attack on what ought 
to be taught. There is, of course, so much hypoc- 
risy in the whole field anyhow that even to get 
people to be a little more honest would be helpful. 
(“The purpose of my course is to teach the stu- 
dents to think"; but the final examination consists 
of multiple-choice questions about the facts in the 
subject matter.) Very few determined attempts 
have been made to formulate instructional course 
objectives so as to reflect in the final examination, 
as well as in the development of the course, what 
the professor professes is really the most impor- 
tant objective of his educational endeavor. Some 
hopeful steps in this direction have been taken. 
(The last course I taught got al most to first base 
in this respect, and the next time I teach it I hope 
Ican get a little further along.) 


EMPIRICALLY GUIDED PRODUCT DEVELOPMENT 


Next to a determined attemptat research on what 
Should be taught, I would say the next most impor- 
tant priority within the total research and develop- 
ment sphere is not in the research field proper but 
in the area of development. In my opinion, a great 
deal more support thanis presently the case should 
be devoted to empirically guided development pro- 
jects. (The U.S. Office of Education has not as 
yet fully taken my advice on this point.) Perhaps 
one way to proceed would be to require product de- 
velopment as one element of an R and D package. 
That is to say, if one is to conduct an experiment 
which purports to ‘‘test а hypothesis” and derive 
some ‘‘fundamental knowledge," and if one is 
realistically pessimistic about the likelihood that 
he will succeed in this objective, perhaps he might 
be asked to have a second string to his bow. If, 
for example, all such experiments were conducted 
with actual subject matters taught in schools and 
did both of these two things, then if they failed in 
what was the nearest and dearest objective to the 
heart of the investigator, they still would have 
accomplished something useful. That is, if one 
produced a useful package of demonstrably rea- 
sonably effective new instructional materials that 
were needed as a vehicle for the experimentation, 
and if this product of the research wereto be made 
available to some appropriate distribution agency, 
then the sole product would not be confined to a 
dubious conclusion in the pages of a journal; the 
product would include an actualon-the-shelf in- 
Structional item that could be used. If in addition 
a little illumination was shed on the basic dynamics 
of instructional variables, so much the better. If 
it weren't, at least one would have a useful film, 
self-instructional program, book, etc. which 
would help to justify the investment that went into 
the research project. 


Also, some studies could usefully take an exist- 
ing product, widely available and in current use, 
and use it in a research project as the vehicle for 
making experimental manipulations to inquire 


about the effects of instructional variables. Then, 
data giving a good profile on the actual teaching 
accomplishments of this product could be made 
available in the form of product-specification data, 
This would thus provide a well-defined set of meas- 
ures, analyzed item by item, so that one would 
know which points of information, skill, knowledge, 
or other behavioral outcome were increased and by 
how much by the use of the basic product in a speci- 
fied manner with specified kinds of learners. One 
would then at least have some useful product data, 
regardless of whether or not his attempt at scien- 
tific inquiry on instructional variables were success- 
ful. 


In summary, I would tend to plug first for em- 
pirically-guided development studies, second for 
other evaluative studies, and (a low third) for purely 
**basic'' research, given the present state of the 
art. A good way to have one's cake and eat it too 
here is to do studies which combine investigation of 
instructional dynamics with the development or 
assessment of a product, in such a way astocreate 
either a useful product or useful product data, in 
addition to whatever illumination may be shed on 
more fundamental, generalizable problems. 


SOME WEAKNESSES IN''BASIC'" RESEARCH 
“FINDINGS” 


As to the frequency or likelihood of genuine i] 
lumination, suppose that one applies the criteria 
any real scientist has the right and obligation to ap- 
ply to research findings, including that of replica- 
tion; and that one throws out all of the findings 
which lack any adequate guarantee of the sensitivity 
of measuring instruments or fail to show sophisti- 
cation about power of statistical tests used, and 
that one throws out all of those indefensible findings 
which have proven the impossible by violating the 
injunction against proving the null hypothesis. One 
then gets down tothe relatively few findings in which 
significant differences are reported, of which he 
will still find that very few have been replicated, 
Of the small number of these, perhaps most are in 
the field of the role of active response of students. 
Even here, however, it is difficult to Say much 
more than simply that some forms of student re- 
sponding under certain circumstances can prove 
quite helpful; the conditions under which one form 
or another of responding can help how much are 
still far from clear. Also, there seems to be some 
pretty consistent evidence that we have probably 
underestimated the extent to which we ought to help 
the student out by prompting; but that evidence is 
more secure for paired-associate learning than for 
more involved kinds of instructional mater ials 


A more gross generalization that I think could be 
supported as a kind of by- product is that most in- 
structional lessons, films, television lessons, and 
college courses try toteach more than they possibl 
can in the time that is allotted them, at least on th 
frequent assumption that these are the sole Шейит 
of instruction. Demonstrably, almost every instr > 
tional program as initially presented is found to ne E 
to be extended, to need to be given moretime [os 
else its objectives need to be reduced to more aa 
est size) when the program is tested to reve] 
profile of what students actually learn. This Ean iba 
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found in study after study when the results have 
been reported in such a way that you can find out 
what was actually learned, not just in terms of P- 
value or a misleading dichotomy between *'signifi- 
cant” and nonsignificant findings. 


The conclusion may bea discouraging one, but 
the most important variable in the effectiveness of 
instruction all too often Seems to be simply the 
in instruction, Thus, 
if one wishes to establish the importance of a pro- 
€ than has customar- 
ily been done in ruling out not only the null hy pothe- 
sis but also what some of us have come jokingly to 
call the ‘‘erg hypothesis, ”’ Too often the proce- 


Many of the weaknesses of the instructional r e- 
Search to date are, I think, due to the lack of the 
promulgation and enforcement, for so-called basic 
research studies, of adequate standards of Scientific 
reporting (vhich we have tried to formulate for 
evaluation studies in the Joi nt Committee report 


The generally poor Standards 
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clusions stated as if they follow fro 
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are likely to interact significantly with intellectual 
ability, as measured at least grossly by IQ, Army 
General Classification Test scores, etc. Yet study 
after study appears in the literature in which only 
the overall data are reported, with no indication 
whatever, even though the basic data contains this 
information, of how the results interacted with, or 
were modified by, differences in student ability 
Even when differential analyses are a function of 
ability are made it is often impossible, without going 
back to the basic data, for one study to be compared 
with another which has, let us say, made one split 
on an ability measure while another study has made 
a different Split, or used a different measure, It 


dict or confirm each other, but a person cannot do 

it because he doesn't have the same split on the dif- 
ferentiating variable, This often would be very easy 
to rectify if the basic data were preserved, as also 
would be true in other instances in which the analy- 


Sis has been incorrectly performed but might be cor- 
rected by going back to the basic data. 
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transcended the level of mere verbal statements on 
a questionnaire by the respondents as to how they 
think they were affected or how they believe they 
would behave in a particular situation. Only a few 
studies have gone beyond this. One such study was 
conducted by Mark May at Yale some years ago 
(May and Jenkinson, 1958). Instead of simply ask- 
ing the students, May observed how students actual- 
ly behaved so as to obtain a measure of motivation- 
al influence of two alternative forms of a film. 
George Gropper and I dida somewhat similar study 
a few years ago in which we assessed the effects of 
some television broadcasts inalternative forms pre- 
sented over WQED in Pittsburgh (Gropper and 
Lumsdaine, 1961). Janis and his associates at Yale 
have also used observation of real criterion behav- 
iors, instead of mere verbal surrogates, in study- 
ing the effects of various forms of communications 
(see Janis and Feshback, 1953). 


But such studies represent only a bare start in 
the directionof what needs to be done. That this is 
so leads to a final conclusion I would like to draw: 
namely, Ithink it would be much better if we would 
pickoutarelatively small number of problems, and 
a relatively small number of standard subject mat- 
ters, for such experimentation as purports to get 
into the real dynamics of instructional variables, 
and stick to them and really try to pin a few things 
down by extensive varied replications with good de- 
pendent measures, instead of striving so much aí- 
ter novelty that we hop from problem to problem 
with a series of superficial studies that cannot be 
compared with each other and that really prove 
nothing, individually as well as collectively. I be- 
lieve this can be said, as a matter of fact, through- 
out the whole spectrum of educational- psychological 
research. There are those who advocate, anda 
few who even put into practice, the insistence that 
Master’s theses should always be replications of 
experiments rather than attempts at new experi- 
ments. I personally think that this would bea 
healthy general practice;for instructional research 
I know it would be. We need a great deal more rep- 
lication and elaboration of a few putative findings 
that look fairly promising, and much less uncoor- 


dinated striving after novelty in an attempt to win 
kudos in ‘‘pioneering. ” 
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Defects, and Needs 
* SOCRATES, A Computer- 


Based Instructional System 


in Theory and Research 


THIS PAPER describes a cybernetic 
computer-based instructional System, SOCRATES, 
the teaching model which led to its development, and 


Some of the research accomplished with it. Early 
work on поп-с; 


described in Coulson (1962 
(1962), and Bushnell (1962 


SOCRATES 


The acronym, SOCRATES, 
izing Content to Review And Te. 
jects. It consists of a group o 
(I/O) stations wired to a digita 
relay rack. Itisa computer-based 


The system permits controlled i mple- 
mentation of the mode] as we 


data which could lead to its further development. 

Figure 1 shows the confi, 
called SOCRATES П. 
computer with а 1311 
plus a 1710 Multiplex 
Several student 1/O st; 


guration of the system, 
It consisted of an IBM 1620 
disk for additional st orage, 
and terminal unit link ed to 
ations, or consoles. 


Figure 2 shows so 


me of the individual stude nt 
Vo Stations, eachof 


which displayed 35 mm frames 
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of a film strip with a 1,500 frame capacity and pro- 
vided the learner with fifteen keys for either single 


(selected) or multiple (constructed) response to a 
display (Stolurow, 1966). 


AN IDIOGRAPHIC CONTINGENCY MODEL OF 
TEACHING 


The model of the teaching- learning карана дп 
idiographic contingency conception узш ты, 
1965b). It is based on the assumption that us Ssa 
alized instruction is a multiple stage E 
ess that is diagnostic, prescriptive and co ule de: 
in its use of psychological information to m Mogens 
cisions about teaching. While the decision p tified. 
is basically Continuous three stages can be iden vi 
In the first Stage the pre-tutorial decisions a Й 
made; in the Second, the tutorial decisions are 
made; and in the third, the program revisions are 
made. In the first two stages there are two levels 


of operation which make the system highly adaptive 
to individual differences, 


Pre-tutorial Decisions 
torial Decisions 
In the pre-tutorial Stage, one set of decisions de- 
termines the assignment of each Student to an instruc- 
on; the other set determines the educa- 
Bram to be used in teaching a student. 


tional stati 
tional pro, 


Automatic assignment, When a student entered 
ea, he could sit atany vacant 


ri 
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Figure 1. SOCRATES II 
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(1,500 frames) and each could contain a different 
library (film strip) if this seemed feasible and de- 
Sirable. Feasibility depends, in part, upon the 
amount of core storage one needs for the computer 
programs used to process each of the educational 
programs contained in a library. 


The purpose of the automatic assignment was to 
simplify and expedite the process of getting the stu- 
dent on-line with the computer. The procedure in 
automatic assignment consisted of the following. A 
student went to any station — for example, 11. If it 
was not one at which the instructional materials 
needed were contained, he had to be directed to an- 
other station which had the appropriate material. 
In order for the System to do this automatically, the 
Student read an initial frame containing instructions 
to both enter his ID number using the keyboard and 
to press the “I am thru” key. When he did this, 
another frame appeared on which a film number was 
displayed. Every film contained a unique number, 
After the student entered his ID, he entered the film 
number, The system then compared the number 
with the one it had stored as the appropriate assign- 
ment for that student. If there was a match, in- 
Struction began at that Station. If they did not match, 
however, the system searched all available Stations 
containing the film which the student should see and 
advised him of the one available; it instructed him 
to go there. This was done by showing hima station 
number. Once the student was seated at an appro- 


priate station, the pre-tutorial phase of instruction 
began, 


Initial selection of an individual’s program. In 
order for the system to make pre-tutorial decisions 
about the student’s instructional program, it is nec- 
essary to have a computer program which processes 
information about the educational program and about 
the student. The objectives of the instructional pro- 
gram must be stated in behavioral + 
type and level of performance that 
objective is a “do Statement" 


Each film in- 
) information; (b) direc- 
; (d) response 
j (f) evaluative 


The flow diagram for program selection during the 
pre-tutorial process is presented in Stolurow (1967). 


Selection of contingencies, To use data about the 
student in making the decision about how to teach 
him, it is necessary to have information about the 
typical, or expected, relationship between pretest 
Scores, on the one hand, and the effects of specific 
treatments such as a type of feedback or a sequence. 
on the other hand. Each of these relationships is a 
possible contingency that has to be implemented by 
the system. The desired со ntingencies are ex- 
pressed as **if... then" statements. To illustrate: 
A contingency, as a rule to be used during learning, 
States that if a student with low math a ptitude an- 
Swers correctly, then he is sent on tothe next frame, 
but if he answers incorrectly, then he is sent back 
to the original material and must select another al- 
ternative. Thisisacorrectioncontingency. Another 
example is the aptitude-sequence contingency, name- 
ly, if the student’s aptitude scores are X, then the 
Sequence of instructional material is 1, 2, 3. For 
example, if a student with high language aptitude 
correctly answers the first five questions about a 
passage just read, then the remaining three ques- 
tions in the set are omitted and he is given new ma- 
terials that he would not otherwise see. However, 
if he does not answer correctly, the remaining ques- 
tions in that set are given. The overall instructional 
logic is what determines the order in which the stu- 
dent would receive new materials during learning. 
If research data show that students above a particu- 
lar level of mathematical aptitude learn to add frac- 
tions better through the RULEG Sequence than they 
do through the EGRUL sequence (Taber, et al, , 1965), 
then a contingency between a Plitude level and the 
RULEG sequence would be programmed and used in 
making decisions about teaching the addition of frac- 
tions. Students with scores outside that range might 
be taught with another Sequencing rule, e.g., asyn- 
chrony (Detambel and Stolurow, 1956), or class-de- 
Scriptive cueing (Wulff and Stolurow, 1957). 


To specify another contingency each student's per- 
Sonality test scores can be used to determine the 
form of evaluative feedback informing him of the cor- 
rectness of his response (Frase, 1963; Parisi, 1965). 
These are just two examples of other contingencies i 
that can be used in making teaching decisions. Idea 
ly each contingency should be based upon data from 
research revealing that it contributes to the effec- 
tiveness of instruction. However, if appropriate da- 
ta are not available, a set of theoretical possibilities 
сап be used to specify the type of feedback, the ee 
Sequence, еіс. Whenever the teaching я 
Contingencies, it branches the course of Hu фе ra 
at the decision points. A systemicanimaxe а p 
Sion for each student at any point in teaching and a 
ternative contingencies can be compared in their ef- 
fectiveness to determine the value of branching. In do- 
ing this, the system is being used to test hypotheses 
about contingencies. Thus research needs have 
to be satisfied as well as those of application. All 
rules specifying the initial set of contingencies have 


tus of hypotheses and, as they are used, the 
system acquires the 


ds 
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using a particular set of materials. In individual- 
ized instruction the logic may be quite different for 
different students. The system must be pro- 
grammed to select a strategy to go with the topics 
to be taught each student. The combination of con- 
tent topics and strategy makes up an instructional 
program for a student. 


Automatic adjustment. The system must also be 


able to handle the student for whom no progress 
could be expected to produce the minimum level of 
performance specified by the instructor. 


Whenever the student's data indicate that he 
would probably not reach the instructional objec- 
tives by any available program, the system can a) 
teach the student some things to remove his defi- 
ciencies; b) lower its minimum acceptancecriteria 
for knowledge; c) lower the terminal objectives; or 
d) reject the student. Each of these alternatives 
can be programmed so that it is considered auto- 
matically in terms of a predesignated priority rule. 
For instance, if the decision is to improve the stu- 
dent's proficiency, then he would be given remedial 
instruction in one or more of the prerequisites. For 
example, the student could be assigned and givenan 
algebra program to build up his entry behavior lev- 
el before he begins studying a statistics program. 
When he completed the algebra program, he would 
be tested and his new entry level of performance 
would be substituted for his original level and an- 
other decision would be made about teaching him. 


Another alternative is to change the minimum ac- 
ceptance criteria. The student below minimum- per- 
formance level on the prerequisite knowledge test 
but high in relevant aptitudescan be given an instruc- 
tional program whichis most likely to increase his 
final performance on the test of terminal behavior. 
Doing this would increase the proportion of failures, 
but it also would increase the absolute number of 
successes. Therefore, this is a decision strategy 
that probably would be used only if it was very im- 
portant to meet a quota of trained people. 


As a third alternative, the system can modify 
terminal objectives when it fails to find a suitable 
program. One way to modify objectives is to in- 
crease the maximum allowable instructional time. 
For each change in that parameter, the system can 
re-estimate a student’s final performance to see if 
the next estimate meets the minimum terminal pro- 
ficiency level. Once the estimated score equals or 
exceeds the minimum, the system can print this out 
so that the instructor can decide what to do next or 
it can compare the estimated time value with a new 
maximum value. The result of the comparison de- 
termines whether a program will be used with the 
Student, A second way to modify objectives is to 
eliminate topics that have low priority. A third way 
is to lower the minimum level of performance which 
is acceptable at the end of instruction. Eachof these 
Ways of modifying objectives can be programmed so 
that the appropriate parameters are considered se- 
duentially and a new estimate of final performance 
Obtained. The system can store the estimates from 
each of these modifications and selectthat one which 
Provided the maximum final performance estimate. 


———— 


The system can be programmed tooperate at two 
different levels in making pre-tutorial decisions. At 
the lower level, it makes decisions based upona 
fixed set of rules and a fixed set of values for the 
Critical parameters used by those rules. At the 
higher level, the system changes the values of pa- 
rameters according to a different type of rule, one 
which determines the priority of the parametric 
changes and then re-evaluates the Student. The pri- 
ority rule itself can be variedfrom student to student 
to achieve further individualization. For example, 
the priority rule can be made to depend upon the rel- 
ative size of the discrepancies between the original 
criterion values and each student's Scores. 


The outcome of the pre-tutorial processing is a 
tutorial program for each student based upon data 
about his aptitudes, abilities, and personality. 


Teaching Program 


A teaching program is made upof two parts: its 
logic and its content, the particular topic or set of 
topics to be taught. The complete set of rules used 
in making all of the decisions in teaching a student 
is the logic of his instruction. The content depends 
upon the student's pretest performance on a test 
Covering the behavioral objectives of the instruction 
and his performance while learning. Selection from 
the library depends upon pretest performance. If 
pretest performance on a topic is already equal to 
the minimum acceptable final performance leve 1, 
then the material used to teach it is eliminatedfrom 
the student’s program. Thus, the content of the stu- 
dent’s program depends upona contin gency. His 
program contains only those materials which are 
necessary to teach the knowledge and skills which 
his initial level of performance indicated were in- 
adequate in comparison with the terminal standards, 


The functions involved in individualizing an in- 
structional program are described in Stolurow and 
Davis (1965). Of those represented the more criti- 
cal functions are the comparison of the student’s re- 
sponses with those expected (performance stan d- 
ards), the selection of particular displays, and the 
recording of a response history for on-line use as 
well as for course revision. 


Teaching Strategy 


A teaching Strategy can be described in te rms 
of two types of rules used in presenting a set of 
Írames. The first type of rule deter mines se- 
quence, or order, i.e., organization of the frames 
relating to an objective. The order in which instruc- 
tional materials are presented to the student is de- 
termined by contingency rules relating to aptitude 
patterns. Each element of instruction is presented 
as a frame which requires a response (Markle 
1964). There is a variety of Írames, each of Which 
has a different purpose. The order in which they 
are arranged сап make a difference in rate of learn- 
ing and in retention (Stolurow and Lippert, 1964), 
Consequently, the type of frame andthe Sequence in 
which frames are arranged represent im portant 
specifications for a program. The second type of 
rule specifies the conditions under which the frames 
are presented; for example, a picture which is part 
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of a frame may be timed, or the medium used to 
encode the frame might be a film, for some, or an 
audio tape, for others. A complete set of rules in- 
cludes both types, and the set employed in teaching 
a student to reach an objective is referred to as a 
teaching strategy. For example, a young student 
with high mathematics aptitude might be taught with 
a RULEG sequence under the conditions of the cor- 
rection procedure and with visually displayed 
frames. An older student who is also high in math- 
ematics aptitude might be taught the same concept 
with a RULEG sequence under the conditions of the 
non-correction procedure with audio tape. Each 
would be taught by a different teaching strategy. 


Data for instructional decisions. Inorder for 
the teaching program to be effective, the system 
must use appropriate data correctly. The kinds of 
data to be used are specified by the teaching strat- 
egy, or rules expressed as contingency statements. 
To make individual predictions regarding the final 
performance of each student from his performance 
history, both the data and the rule possibilities have 
to be represented in the computer memory and used 
by the control program. In this respect the system 
operates in the tutorial mode just as it did in the 
pre-tutorial mode. 


If the predictions about final performance are be- 
low the desired level, the system should be capable 
of changing the rules it is using and the basis upon 
which the rules are selected. For example, regard- 
ing the latter point, the system after teaching se v- 
eral students using a particular teaching logic, 
should use the data they provided it to ге - analyze 
and compute new limits. 


Whenever the student's performance does not 
meet the specified standards, adjustments are made 
in some part of his projected program. Also the 
data contribute to the adjustments made in the deci- 
Sion rules themselves for future applications to 
Other students. In this way his data contribute to 
the pool of information used in making decisions 
about teaching Subsequent students (e. g., Smallwood, 


1962; Lippert, 1967). As а result of its previous 


teaching, the system lea: Е 
fectively, y rns how to teach more ef 


Two Levels of Tutorial Instruction 


Two levels of tutor 


tent, e.g. ч ith a fixed logic and а fi 
and practice (Atkinson on! 1954), drill 


tlhe ] The higher level ob- 
t ag System is programmed toalter ether 
1с, Ontent, or both, for Subsequent parts 
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A distinction can be made between a pre-specified 
learning environment and a learnin g environment 
that unfolds. In the former, the actualdisplays 
seen by the student are specified in kind and in se- 
quence. If the student makes response “A,” then 
he sees display o, etc. The system operates on the 
basis of conditionals that are of the form ‘‘If this re- 
sponse, then that stimulus display.’’ All the condi- 
tionals are specified, and all students who make re- 
sponse ‘‘A’’ see display œ. In the learning environ- 
ment that unfolds, all students who make response 
**A" do not see display œ, only some of them do. 
What display a student sees is determined by more 
than one rule, so that display a, for example, 
could be seen by some students who made response 
СА? but not by others who also made that response. 
An example of unfolding might be to show some stu- 
dents a frame containing the steps in the long divi- 
Sion algorithm worked out with the values 360 divid- 
ed by 12, while others may see the steps of this al- 
gorithm worked out with 480 divided by 52; some 
others may see examples with decimals. Still oth- 
ers may see the example worked out ina completely 
different sequence, e. g., successive subtraction and 
tallying. To do this on some basis other than chance 
implies that the decision is based upon more infor- 
mation about the student than that contained in the 
last response. This can be done by making available 
a response history of two, or more, previous re- 
sponses. To do this a CAI system must be capable 
of keeping a cumulative record of res ponses ina 
form that makes the pertinent set available eye ry 
time a contingency rule is to be used, 


In addition to the availability of response histories, 

the CAI system programmed for unfolding a learning 
environment also must store in readily available 
form the rules for sampling the frames and the me- 
dium (e.g., audio or visual) or media to be used. 
Another requirement is the ability to use some con- 
tingencies, but not all, witha particular student. 
Related to this is the requirement for the system to 
be able to change the contingencies for each student 
if his performance is not meeting expected stand- 
ards. This is called the feed forward concept in 
controltheory. The system looks ahead to expected 
results based on what has happened and what possi- 
bilities could eventuate. After looking at the d 
possibilities it selects the most desirable one ап 


then proceeds to do those things necessary to bring 
it about. 


One type of rule specifies the sample of Wen 
In a linear program the sampling rules pos ИП eee 
all students take the same program and t е rhe sam- 
the same set of frames in the same R 
ple rule is to use the entire set of frames. 


In idiomorphic programming the enc ig ied 
ment is predetermined but not pre-specifie Я ие 
determined learning environment can be generate 
in either one of the two different way S referred to 
in the previous paragraph or by using both ways. 
Each way uses conditional rules of the form, ‘‘If re- 
Sponse ‘A,’ then use procedure o to determine the 
next display."' The display that appears next is pre- 
determined by a procedure which can generate not 
Just one, but a number of different displays. 


One way to implement this form of conditional is 
to use a procedure that varies the content (e. g., the 


Fi 


SOCRATES, A COMPUTER- BASED INSTRUCTIONAL SYSTEM IN THEORY AND RESEARCH 107 


numerals) but not the sequence (e. g. , thealgorithm). 


A second way to implement it is to use a procedure 
that varies the sequence, but not the content. A 
third way is to vary both. The first way makes the 
sequences or logic conditional; the third way makes 
both content and sequences conditional. 


Another type of instructional strategy rule deter- 
mines the time intervals. With a pre-specified 
learning environment of the linear type the strategy 
rule is simple since all students go from frame 1 to 
frame 2, to frame 3, etc., and in the same order. 
The rule is simple but the value can vary since the 
time intervals are determined by the student; he 
sets his own pace. By pre-specifying the conditions 
the only individualization achieved by a linear pr o- 
gram is in the rate at which each student progresses 
through the material. By pre-specifying the out- 
come of each response in an intrinsic program, all 
the branching possibilities are fixed, and each re- 
sponse alternative takes a student to a specific dis- 
play. The contingency that is pre-specified in this 
form of programming is that between the last re- 
sponse and the next display. 


The learning environment can be predetermined 
in still other ways. For example, the content of the 
next frames may be different depending upon the 
correctness of the student's responses to other 
frames on the same topic, and their sequence may 
depend upon aptitude scores. 


With the predetermined level of functioning, the 
on-line decisions made by a teaching system change 
one, or more, of the teaching rules, or theteaching 
strategy. Two students, each making the same re- 
Sponse to frame 105, for example, might see differ- 
ent displays, experience different ti me intervals, 
and have information available for different lengths 
of time, etc. The information display, the sequence 
of displays in the set, and the timing conditions 
would be contingent upon specific information about 
the student's past performance. One question to be 
answered is what type of information can be used in 
making decisions. This isa research question to 
be answered ultimately by empirical data. Hypoth- 
eses can be formulated; for example, it is possible 
to include aptitude and personality test scores as 
well as sets of students' responses to instructional 
frames during learning in the **if" portion of the 
conditional statements. Whichever one, or combi- 
nation of scores is used in specifying contingencies 
for instructional decisions will depend upon the ed- 
ucational programmer's hypothesis regarding the 
optimum strategy of instruction. 


When a computer-based system provides a con- 
tingency-determined learning environment ba sed 
upon individual differences in performance, it uses 
idiomorphic programming. In teaching a set of con- 
cepts the rules which determine the order in which 
they are presented may change, for example, from 
an inductive sequence to a deductive one. While any 
rule may be changed, it might be useful to consider 
examples in which only one is changed. For in- 
Stance, only the rules determining the information 
Sequence might be changed. Another example might 
be the rule specifying the evaluative feedback c on- 
tingencies (given the correctness of the learner’s 


responses). If the student’s performance is not up 
to expectation in correctness or in response latency, 
the system may change the rules used to teach him. 
For example, the rule being used may have been to 
give the student reinforcement every time he re- 
sponded within a certain period of time. If he did 
not respond quickly enough, the reinforcement may 
be given on only half of the times he responded. 
This type of teaching rule would specify the contin- 
gencies between response latency andthe subsequent 
reinforcement schedule. If the levelof the student's 
performance is below that expected, easier frames 
may be presented to him. This last example illus- 
trates yet another type of rule, one making the dif- 
ficulty of the subsequent material contingent upon 
the number of errors. 


These examples illustrate the types of decisions 
that can be made when contingencies are predeter- 
mined but not pre-specified. Instruction is more 
individualized when instructional rules are contin- 
gent upon both the student’s performance and what 
is expected of him. Whenever one or more aspects 
of the student’s actual performance do not match ex- 
pected performance, a new rule isused. When a 
new rule is used, sets of displays and/or frame se- 
quences become available that would not be available 
if the system operated at the pre-specified level of 
a linear or an intrinsic program. 


Bases for Contingencies in Instruction 


This analysis of teaching as the control of the 
learning environment is based upon the idiographic 
contingency model of tutorial instruction (Stolurow, 
1965b, 1965c). This model distinguishes two types 
of contingencies. At the lower levelthere are event 
contingencies; at the higher level there are rule 
contingencies. The model is made operational in 
SOCRATES in three ways. 


Student data possibilities. The first is inthe 
number of possibilities which the system provides 


for the use of data in making decisions concerning 
the content and strategy tobe used in the instruction 
of individual students. For instance, the student’s 
ability, aptitude, and personality, as wellas his 
knowledge or skill, can be used in making decisions 
at any point in the program. Patterns of responses 
that is, various patterns of error andtime, made. 
during learning are included in the possible student 
characteristics which the system can use. Sets of 
these patterns are possible bases for making the 
subsequent experiences contingent upon past per- 
formance. It is possible to use pretest perform- 
ance along with error and time dataon selected 
frames as the basis for content and/or sequential 
contingencies. Consider an example of how these 
data might be used to make different decisions for 
each of two students. One student may have made 
a high pretest score and then gone through the pro- 
gram rapidly, making lots of errors; another stu- 
dent may have made a low pretest score, but gone 
through the program in the same way. The first 
student might be assumed to be careless, and, there- 
fore, given instruction designed to make him more 
careful. The second student could be assumed t 
have failed to learn because of insuffici 9 
ability and therefore, to be in need of 2 
plementary or remedial in Struction, SUBE 
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Different display possibilities. The earlier ex - 
ample in which the system provided different conse- 
quences to students who made the same response to 
a particular frame illustrates a second wayacom- 
puter-based system can make instruction more in- 
dividualized: namely, by providing different display 
possibilities for every pedagogically- significant con- 
tingency. The system's adaptivity is increased by 
including in its library a large number of possible 
displays for each response possibility. The number 
of possible displays is increased in different ways. 
One way is by writing sets of materials at different 
levels of difficulty, another is by using different 
forms of encoding to communicate information, e. g. , 
verbal versus mathematical. A third way is to sep- 
arate the informative material thatis intended to ex- 
plain the answer from the different kinds of evalua- 
tive feedback. The example of the student who was 
responding carelessly is a situation in which the 
availability of separate types of feedback would be 
useful. With the separate types available in this 
System, it would have been possible for the system 
to tell him he was careless while also te lling the 
other student who made the same response that he 
was improving. Furthermore, each student could 
be given his Subsequent instruction with another 
teaching strategy. For example, the careless one 
could be required to make Correct responses before 
being allowed to go on. The other student in the ex- 
ample, however, could be branched to d ifferent 
evaluative feedback, possibly of an encouragingtype, 
and then given easier material so he would have а 
higher probability of being correct. 


Either an instructional theory or empirical data 
specifies each of the relationships between a set of 
Student characteristics and a set of display possi- 
bilities; consequently, the more display possibil- 
ities in the memory of the system, the more contin- 
&encies it can handle and the more individualized 
the instruction could be, 
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terval between knowledge of results and the new 
program frame. 


Criterion possibilities. A third way in which idio- 
graphic programming can make a system more 
adaptive is by using a variety of criteria in making 
decisions to either maintain or change the instruc- 
tional rules themselves. A flow diagram describing 
a system for doing this is presented in Figure 3. 


The program which allows the system to change 
rules on line is called the professor function. Bas- 
ically this is the means for determining rule contin- 
gencies. It is to be contrasted with the ability of the 
System to function according to a fixed set of rules, 
which is called the teacher function. The teacher 
function is the part of the executive program that 
specifies the conditions for showing each one of a 
set of displays, The professor function is the part 
of the executive program that specifies the condi- 
tions for showing one set rather than another. These 
two functions are represented in Figure 3. 


The professor function also requires ‘‘if.., 
then..." rules, but these are rules that make the 
use of lower order rules contingent upon specific 


Figure 3. System of criteria for decision- making 
in idiographic programming. 
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conditions. The professor rules imply teacher 
rules. In other words, if a student performs either 
better or worse than expected when a particular 
teaching strategy has been used, then some, or all, 
of these rules would be changed. This means that 
at least one other teaching strategy must be avail- 
able as an alternative to the one being used. When 
an alternative is used, frames may bedisplayed that 
would not be displayed if the previous strategy had 
been used continuously. 


An example of a pair of strategy rules which 
could be used by a system on an either or basis 
would be correction procedure with an inductive se- 
quence rule or with a deductive sequence rule. In 
using these strategies, it is necessary to have the 
conditions specified and a criterion available so that 
it is possible to determine if the student failed to 
meet the performance criterion. Assuming that he 
failed to meet the criterion when he was taught by 
the inductive strategy, he would be shifted to a de - 
ductive strategy in the next set of frames. Inac- 
complishing this change, the criterion for the pro- 
fessor function might be ten consecutive correct re- 
sponses none of which takes more than half a minute. 


Whenever the student is being taught by one strat- 
egy rule, the teacher function is in control. How- 
ever, the shift from one teaching rule to another is 
accomplished by the professor function which be- 
comes effective whenever student responses fail to 
meet a criterion. Every time the student's perform- 
ance fails to meet the professor criterion, one or 
more rules of instruction are changed. If a teacher 
criterion is not met, then the decision that is made 
is specified by a particular rule. The difference be- 
tween the teacher and professor functions can be 
made more explicit by indicating that the use of one 
teaching rule may utilize a fixed set of instructional 
and feedback frames whereas the useof tvoor more 
rules may utilize additional informational and/or 
feedback frames. Therefore, with an executive pro- 
gram that can make rule changes and use different 
criteria for the changes, students may see materials 
when one rule is being used that they do not see if 
another one is being used. 


Two types of criteria can be used in making de - 
cisions about instructional rules. One type of c r i- 
terion is a predetermined value; it may be a value 
for errors, for time, or for a combination of both 
of these. It is illustrated by a conditional statement 
such as *'If the student makes three or more correct 
responses to this set of frames, then continue with 
the use of rule x." The other type of criterion is 
generated by an algorithm, or computational formu- 
la, on line. This latter type of criterion is not pre- 
determined; rather the algorithm and the nature of 
the data it uses are predetermined. Student re- 
sponse data are processed as they occur and the re- 
Sult of the processing is the criterion value used in 
deciding to continue or discontinue usinga rule. The 
following is an example using response rate. lí the 
Student's rate of response increases so that the val- 
ues fit a linear incremental function within a speci- 
Hed range of variation in slope, the strategy that is 
being used to teach him is maintained. If his rate 
of response does not fit a linear function within the 
range of permissible variation, the strategy shifts 
to the use of stronger positive evaluative feedback. 


Another example of a computed criterion is the use 
of a mean value based upon the performance of stu- 
dents who have just been taught. As each student is 
taught, his data are used in the re-computation of 
the mean and standard deviation which become the 
new criterion in making a decision with the next 
student. In this sense the system has a me mory, 
for it accumulates data on students it teaches and it 
adjusts its future performance based upon the way 
in which it is programmed to use the accumulated 
data. These are some of the ways in which 
SOCRATES can utilize response history andcriteria 
to determine its future operation. Thus it functions 
as a cybernetic system. A complete record of each 
response and its latency is available. These data 
can be summed across topics or across students as 
desired to obtain a performance measure that is 
read out or used internally by the system for educa- 
tional or research reasons. Furthermore, all rec- 
ords are kept for a student rather than for a station, 
In other words, a student can move fromone station 
to another and the system keeps all his records to- 
gether. He can stop and start up again at any time 
and the system has immediate access to all of his 
test scores and past responses that it was pro- 
grammed to save. 


IMPLEMENTATION OF THE MODEL 


In order to accomplish the teaching functions of 
the idiographic contingency model, it is necessary 
to prepare instructional materials, including tests, 
and to write computer programs to manage the in- 
structional materials in order to collect the data 
and to process it. 


LOADING THE SYSTEM 


The preparation of instructional materials 
for use in SOCRATES involves two processes. One 
is photography; the other is computer programming. 
The student looks at photographed images of printed 
or drawn materials. These can be relatively con- 
ventional programmed instruction frames, since 
pre-tutorial decisions can be based either upon ex- 
tant standardized tests of achievement, ability, ap- 
titude, and personality, or upon specially- prepared 
tests. A convenience of SOCRATES, which does 
not exist for some other computer-based systems, 
is that any test material prepared in conventional 
form can be photographed and used on the system. 
Processing materials to make them actually avail- 
able under computer control can be accomplished 
relatively quickly since they simply needtobe photo- 
graphed and then a program can be written to con- 
trol the duration and sequence of their display. The 


students’ responses are recorded and scored auto- 
matically. 

When materials are loaded on SOCRATES there 
are two parts to the content of its library, 
the displayable information stored on 35 mm. fil 
for which there is a maximum reel capacity ofl si 
frames; the other is the program in the memory f 
the computer, both core and disk, in which th Ü o 
structional strategies and student data are stor in- 
It is necessary to write a computer program s ed. 
trol the presentation of each frame of fil Es EUR 
process any data to be used on-line. With n to 
based systems that use a CRT display, ime а 
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is necessary to write programs which generate the 
test items and the displays of program frames. 


A more interesting use of SOCRATES occur s 
when educational programs are prepared especially 
for the system. It is possible to introduce a great 
deal more variation in the logic of instruction with 
the system than otherwise would be possible. Fig- 
ure 4 presents a set of very simple flow diagrams 
associated with some research conducted with pro- 
grams especially prepared for SOCRATES (Merrill 
and Stolurow, 1966). It also illustrates some of the 
discriminations that are made among materials to 
provide opportunities for flexibility in programming, 
P frames are those frames which represent pro- 
grammed materials, F framesare feedback frames. 
С frames are frames which correct the student. GR 


frames are frames which provide for general re- 
view of the point covered in the Pframe, and SR 


frames are frames which give specific reviews to 
the student. By providing these different types of 
frames in the film library, it is possible to route 
the student through selected subsets of them in what- 
ever sequence seems most desirable, A computer 
program must be written to do this. It would be 


written to provide for the contingencies of a set of 
instructional rules, 


The upper left panel of Figure 4 isa flowdiagram in 
which a set of these frames is used to provide cor- 
rection and review ina lesson. By correction proce- 
dure, we mean that the student must make a correct re- 
Sponse to the material covered inthe P frame before 
he is allowed to go on to the next P frame. If, how- 
ever, it is desirable to route him through a review 
frame on this same material when the student re- 
sponds in a way which the educational programmer 
feels indicates a review, the computer program is 
written to do this. The no-correction procedure is 
illustrated in the upper right panel. Inthis case 
the student is allowed to go on to the next frame 
Whether he is right or wrong; however, he is first 


Shown an F frame which tells him he is either cor- 
rect or incorrect, The use of a no-correction rule 
is typical of a linear programming Strategy. 


now replaces th 
and Ө frames о 


the students. The loca- 
9n to each other on the 

Ocate quickly the frames 
is ашо iu Тате, The reason for this 


1 between the resentati 
and the retrieval of the ext a 


The de 
Strategi 
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Figure 4, Program flow diagrams illustrating the 


experimental conditions of correction. review and 
no correction/review on lessons and quizzes. 


INTR. 
F 
с 


NO CORRECTION/REV LESSON 


CORRECTION/REV LESSON 


INST. 


@) 
yal 


NO CORRECTION/REV Quiz 


CORRECTION /REV Quiz 


of contingency rules is used to provide the instruc- 
tional strategy and a set of one or more strategies 
is used to make up the teaching logic, then the Sys- 
tem is more or less adaptive to individual student 
needs depending upon the extent to which the partic- 
ular set used to teach a Student was determined be- 
fore the student started. If the set was determined 
prior to instruction, the System is not as adaptive 
as if it were determined while he was learning and 
depended upon his performance during learning. 


The research problem in making the system 
adaptive in an effective way is to determine the con- 
tingencies to use and the criteria so that the co m- 
puter-based instructional system is programmed to 
implement those contingencies that instruct most 
efficiently. This is the primary problem in the fur- 
ther development of the idiographic contingency 
model. In order to solve this problemi it will be 
necessary to begin by using the systems in a less 
adaptive level than they are capable of performing. 
Students with different characteristics w ill be as- 


Signed to strategies to determine their relative per- 
formance, 


Code line of library table. A code line of a li- 
brary table is presented in Figure 5. It represents 
9ne of the two sets of computer materials used to 
Control instruction. The library table consists of 
lines of code which determine the frame locations 
and the various legal buttons which the student may 


| 
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Figure 5. A code line from a library table. 
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use in responding to displayed material. The illus- for the design of adaptive learning environments, the 


trated code line presented in Figure 5 isfor instruc- 
tional frame number 7 of a program. This is indi- 
cated on the left by the number 0007. Associated 
with each instructional frame is a setof lettered al- 
ternatives each of which is specified by a rule that 
is part of a teaching strategy. Each letter in the 
code line is associated with a frame address to 
which the student will be taken if he makes that re- 
sponse. If, for example, he makes the correct re- 
sponse, ‘‘C’’ in this example, the film is moved 
eleven framed forward. ''C'" іѕа соггесі response. 
The correct response alternatives are always listed 
before the incorrect ones, and after the first one 
the other correct alternatives are starred. There 
is only one correct response in the example. After 
“С,” the letters “B” “А” “R” appear. Eachis an 
allowable incorrect response. И any other key is 
used it results in no change in the display. 


In addition to the library table there is the co m- 
puter program which actually manages the move- 
ment of the program frames and the times ої 
The various strategies and levels of adaptivity pre- 
viously described can be accomplished through the 
use of these two sets of materials. 


RESEARCH 
Some of the research which relates to the idio- 


graphic model follows. "While the model specifies 
the types of contingencies that appear to be sufficient 


f display. 


exact nature of the contingencies which will maxi- 
mize the rate of learning remains to be determined 

by research. Some studies which have provided in- 
formation about contingency relationships can be de- 
Scribed briefly. Since they are insufficient to docu- 
ment all of its facets, the studies mainly provide 

support for the general validity of the model. 


THE EFFECTS OF SEQUENCE UPON APTI 
REQUIREMENTS FOR LEARNING тири 


A matrix of inter-correlations bet i 
ular set of aptitude scores and the Enc pocos 
sulting from a learning experience indicates the "ia 
tent to which the gains can be predicted from th 
various aptitude scores (DuBoise, 1962). The si E 
nificance level of each correlation indicates lieti 
the aptitude is or is not a significant factor in det i 
mining the gain score which students таке. If aa 
dents are randomly assigned to different tea с а 
Strategies, and a single set of aptitudes is ге], UE 
to the gains made, then differences in the cor epe 
tions indicate that an aptitude is involved in le: ae л 
under one strategy but not under another Thisteoe 
of information relates to the diagnostic ав ы 
model in that the correlations would iden ae ths 
titudes that could be used in assigning | ae 
the instructional strategy that will r esult PAP i 
greatest gains. H the Strategies consist PA 
that determine the frame sequence, then the ood 

, ents’ 
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aptitudes could be used to determine their optimum 
sequence. 


One characteristic of teaching materials which 
can be made contingent upon aptitude isthe sequence 
of frames presented to the student. If studies show 
that the size of the correlations between particular 
aptitudes and learning scores varies when different 
sequences of P frames are used so that some stu- 
dents do well when they learn with one s equence 
while others do well with a different sequence, it can 
be assumed that there is a rule that determines the 
sequence which would be best to use for a student 
who is relatively high in a particular aptitude and 
relatively low in another. Research revealing that 
aptitude-gain correlations do change when different 
strategies are used provides support for the idio- 
graphic contingency model. If aptitude-gain corre- 
lations do not change significantly, then this type of 
model would be inappropriate. 


Asynchrony as a measure of sequence. One prob- 
lem that arises in usin g a diagnostic- prescriptive 
approach is the lack of critical information. Criti- 
cal information might be obtained through the use of 
variables in research that can be implemented in a 
selective way. For example, if frame sequence is 
to be manipulated selectively then it is important to 
have the rules for manipulating sequences so formu- 
lated that they can be implemented easily. There- 
fore, the way in which the sequence of frames, or 
displays, is characterized is important. The con- 
cept of asynchrony as applied to critical and non- 
critical stimulus elements appears to be quite use- 
ful for this purpose (Detambel and Stolurow, 1956). 
Asynchrony can be exemplified in the following illus- 
tration involving color, size, and shape. If all 
frames ina program use color, size, and shape of 
à set of objects and the student is learning a concept 
involving one of them, such as color, then color is 
the critical cue, Consider some of the things that 

ppen in going from frame to frame ina sequence, 
The color may change from one frame to the next- 
for example, from red to blue — but the size and 


In this example, not only are color 


ake place j i 
next one, If o; piace dh 


Critical to what i 


Changes but i oe a 
es 


(0000 IER 


Two opposite types of sequences also can exist. 
These are called synchronous sequences. One type 
occurs when both a critical feature and a noncritical 
feature change; the other occurs when n« 
changes, or, in other words, the disp 
These ways of characterizing sequences ha ve been 
found to be useful in organizing materials in several 
studies (e.g. , Detambel and Stolurow, 1956; Stolu- 
row, 1956; Stolurow, Fogel, and Fogel, 1966; An- 


derson, 1966) and recently in Germany by Kotter 
(1965). 


ther 


These concepts are treated nore extensively in 
Stolurow (1956) and in Garner (1962). 


The use of class-descriptive cues. The measures 
of asynchrony, while reliable for a particular frame 
Sequence, do not, however, identify a unique se- 
quence of frames. Using the same set of frames it 
is possible to make up different sequences which 
have the same amount of asynchrony. In other words 
ifa sequencing rule were used byacomputer to gen- 
erate an asynchronous sequence the computer would 
not necessarily generate the same sequence each 
time the rule was used. However, in all cases the 
frequency with which CC and CNC types of sequences 
occurred would be the same. Since this is SO, it is 
useful to differentiate among sequences with the 
same asynchrony value, A way of doing this is to 
use the concept of class-descriptive cue (Stolurow, 
1956); any feature that does not change from one 
frame to the next frame within a subset of two or 
more frames of a program can function as a class- 
descriptive cue. Therefore wherever there is more 
than one feature that can be useful to the learner, 
the frames should be organized into subsets with a 
class-descriptive cue that is related to one of the 
objectives of instruction, The data thus far collect- 
ed indicate that the use of cla 55-еѕсгірііуе cues in 
organizing displays does facilitate learning (Stol- 
urow, 1956; Wulff and Stolurow, 1957) 


The effects of Sequence changes on the relation- 
ship between aptitudes and gains. An asynchrony 
rule in combination with a cue-organizing rule can 
produce sequences that provide cla ss-descriptive 
cues for the learner, This teaching logic appears 


to be a useful way to synthesize a pr ogram of in- 
struction, 


Two studies will be reported here to indicate the 
results of this approach. In both studies analyses 
were made of the data to determine the resulting re- 
lationships between aptitude and gain scores when 


different Sequences of the same frames were used 
for instruction. 


The first study was conducted by Cartwr ight 
(1962). He used a fractions program which was pre- 
sented to mentally-retarded students during second- 
ary-level instruction. Two variations of the instruc- 
tional material were prepared. 


Inone Program 
there was a Systematic increase 


and decrease of the 
tor of the fractions 
ntfrom frame to 
which was made 
; the sequence used did not 
P. The comparison of the 
€ between a program with a 


ay is repeated. 
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TABLE 1 


CHANGES IN CORRELATIONS BETWEEN APTITUDE AND LEARNING SCORES FOR DIFFERENT 
SEQUENCES WITH EQUAL AMOUNT OF ASYNCHRONY (CARTWRIGHT, 1962) 


Sequence with Systematic 
Increment and Decrement 


Individual 
Characteristics 


Sequence with Non- systematic 
Increment and Decrement 


Mental Age - WISC 
Verbal 
Performance 
Total 


Mental Age - Binet 


Language Aptitude 


(N - 20) (N - 16) 
.02 -.26 
26 -.50* 
17 -.49* 
.18 -.49* 
2 -.54* 


*p<.05 


class-descriptive cue and one without a class-de- 
scriptive cue. Some of the data are reported in 
Table 1. 


Cartwright found, for example, using a total in- 
telligence test score from the WISC, that а. 17 cor- 
relation existed with finalachievement or gain 
scores under the class-descriptive cue (CDC) condi- 
tion; whereas a correlation of -. 49 was achieved 
with a sequence that did not have aclass-descriptive 
cue. With the Binet intelligence test scores com- 
parable correlations were obtained, na mely, .18 
and -.49. Even more interesting is the pair of cor- 
relations with the verbal aptitude test: a positive 
correlation of . 12 was obtained with the CDC se- 
quence and a -. 54 with the other sequence. These 
data indicate that verbal aptitude becomes more or 
less significant as a contributor to learning depend- 
ing upon the sequence in which theframes of the 
program are presented. 


The difference in the correlations for the two se- 
quences indicates that it is possible within a 30- point 
IQ range, from 53 to 83 in this case, to predict from 
either the general intelligence measure or the ver- 
bal aptitude measure what the residual scores of 
students will be under the non-CDC sequence. This 
means that with the CDC sequence neither intelli- 
gence nor language aptitude plays a significant role 
in determining the residual scores within this range 
of ability. With another sequence, however, these 
abilities played a more important part in determin- 
ing what the students learned. 


A second study, dealing with sequence differ- 
ences and also employing a fractions program, but 
a different one, was conducted with normal students 
in elementary schools in Champaign, Illinois. This 
study was completed recently and revealeda pattern 
of relationships between aptitudes and learning as a 
result of sequence differences which was similar to 
that reported in the Cartwright study. Table 2 sum- 
marizes the set of correlations between pairs of var- 


iables which differed significantly for the two de- 

grees of asynchrony. While there were other sig- 
nificant differences between the effects of these two 
sequences in addition to those shown in this table, 

these differences illustrate the point that changes 

in asynchrony can alter the aptitude requirements 

for achievement. 


The data from both studies support the idiograph- 
ic contingency model described earlier. They indi- 
cate that effective individualization of instruction can 
be accomplished by using teaching rules that offer 
instructional sequences which fit the aptitude char- 
acteristics of each student. The use of sequencing, 
and particularly of asychronous sequencing with 
CDCs needs to be studied further to determine the 
sequencing rules which are so related to the stu- 
dents’ aptitudes that the best match for maximizing 
his performance is achieved. The research prob- 
lem facing us now is to determine the specific kinds 
of relationships one needs to take into accou nt in 
making the best match between aptitudes and rules 
for generating particular sequences of frames, 


THE EFFECTS OF VARIATIONS IN ENCODING ON 
APTITUDE REQUIREMENTS 


Another aspect of a teaching strategy i 

of symbol used to express a us of ee orp rice 
other words, the encoding employed tocommunic Т 
the concepts to students. A recent study (Stoluro: + 
Frase, and Odell, 1966) of encoding difference. rei 
ployed two different sets of notation in teachin, i ae 
to college students. One program used the Pe nee 
Russell notation system, the other the Polish nota- 
tion system. In all other respects the two б: 
grams were identical. Thus, any variations Vot 
come could be attributed to the encoding Фен 


The first set of results (Table 3) 
clearly that under the Peano-Russel 
American students perform in a man 
dictable from a set of aptitude tests, 


indicates rather 

notation system 

ner that is pre- 
Comparable 
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TABLE 2 
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CORRELATIONS BETWEEN IQ AND PERFORMANCE AND BETWEEN SPECIFIC AP TITUDES AND 
PERFORMANCE FOR DIFFERENT DEGREES OF ASYNCHRONY (UNPUBLISHED STUDY) 


Variables Sequence with 


Less Asynchrony 


Sequence with Higher Asynchrony 


California Test of Mental Maturity 
Total IQ and Gain Score 


+. 01 +. T3* 
Non-verbal IQ and Gain Score 13 +. 55* 
Iowa Test of Basic Skills m 
Spelling and Gain Score -.32 +. 56 
Stanford Achievement Test 2x 
Arithmetic and Gain Score -.46 +. 70 
* and **: The two correlations are significantly different from each other; *p <. 05; **p« .01 


TABLE 3 


CORRELATION OF DIFFERENT APTITUDE: 
CONTENT TAKEN AFTER LEARNING THE 


S WITH PERFORMANCE ON A TEST COVERING IDENTICAL 
POLISH OR PEANO-RUSSELL NOTATION 


Aptitudes for 


Inductive 
Group Vocabulary Mathematics Reasoning Inference 
Peano- Russell 27* 2191Ж .21+% .01 
Polish . 09 .18 =, (Od ОЛ 
*p <. 05 **p«.,.01 


decode the Symbols represent- 


training produces, 
that the Peano- Russell notatio: 


oo НА 


to the Polish system for students trained in 


proc- 
essing the English language. 


THE EFFECTS OF VARIATIONS IN EVALUATIVE 
FEEDBACK ON PERSONALITY REQUIRE MENTS 


Still another set of contingencies pertaining to the 
model are those relating personality characteristics 
of the learner to his performance gains when differ- 
ent conditions of evaluative feedback, or social re- 
inforcement, are employed. Social reinforcement 
in these studies means evaluative feedback state- 
ments which tell the student about the quality ag 
well as the correctness of his response. Thus, ap- 
proval could be expressed in evaluative statements 
such as, ‘‘Excellent, you are coming along fine ^ 
or ‘‘Right on the button"; for negative evaluation 
or reproof, evaluative statements suchas “No you 
аге not thinking,” “No, what a silly mistake to 


make” express the attitudes, 


Each of these ki; 


nds of Statement. i 
different groups of Жен CUN 


Students all of whom Studied the 


т 
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TABLE 4 


CORRELATIONS BETWEEN PERSONALITY VARIABLES (a) PERFORMANCE, (b) ATTITUDE AND (c) TIME 
UNDER FOUR DIFFERENT CONDITIONS OF SOCIAL REINFORCEMENT (FRASE, 1965) 


same basic program in logic (Frase, 1963). Group 
one (+) had correct and incorrect responses ev a l- 
uated. Group two (+) had only their correct re- 
responses evaluated. Group three (-) had only their 
incorrect responses evaluated. Group four (0) had 
no evaluation of either one, just knowledge of results. 


In Table 4, three sets of correlations are of in- 
terest. One set is between personality test scores 
and learning scores (e.g., aggression/perform- 
ance); a second is between personality scores and 
attitude (e.g., aggression/attitude); and a third set 
is between personality and learning time. All three 
sets revealed that each of the four experimental con- 
ditions of social reinforcement was most suited to 
a different personality group. Three personality 
characteristics, aggression, deference, and intra- 
ception (Murray, 1938) were found to be important 
in relation to performance under the four different 
conditions of social reinforcement. For example. 
students who were high in aggression performed 
best when they were given negative evaluation of 
their wrong responses and not given positive evalu- 
ation of their right responses. Under two condi- 
tions - number 2, positive evaluation, and number 
4, no social reinforcement – aggression was related 
negatively to performance. In other words, for the 
students high in aggression, performance was im- 
paired under these conditions. Under condition 
number 3, negative reinforcement, aggression was 
related positively to attitude but unrelatedto achieve- 
ment. But the relationship with performance, al- 
though positive, was not significantly different from 
zero. 


Under the punitive conditions (number 3) high 
deference was related to lower attitude but was not 
related to achievement performance. 


Intraception was negatively correlated with speed 
and efficiency, and related to high attitude under 
maximum social reinforcement, condition number 1. 


In another study by Parisi (1965) the basic para- 
digm of the Frase study was replicated and an addi- 
tional personality test, the Cattell 16 PF, was given 
prior to learning. All materials were administered 
in Italian to Italian students in Italy. These results 


Aggression/ Aggression/ Deference/ Deference/ 
Group Performance Attitude Attitude Time 
Condition 1 + .15 .24 -. 05 .07 
Condition 2 « as -. 50* -. 06 .48* 
^ Condition 3 - .35 ,mg*es -.59* wot 
Condition 4 0 -.50* -.16 . 00 -.40 
*p«.05 *жжр < . 005 


support those of Frase that negative evaluations 
were more important that positive evaluations. Fur- 
thermore, positive reinforcement was more effec- 
tive than negative reinforcement in reducing the cor- 
relation between intelligence and performance. Per- 
formance on the achievement test was related to 
deference, or lack of need for autonomy, when so- 
cial reinforcement was used. When no social rein- 
forcement was used, performance was related to 
need for achievement and exhibitionism. 


The data from these studies seem to show that 
it is possible to alter the quality and speed of the 
performance as well as the attitude of students by 
the way in which social reinforcement is used during 
the learning experience. They further suggest that 
it might be possible to achieveanoptimum matching 
of personality characteristics with social reinforce- 
ment conditions for each individual student so that 
the general level of effectiveness of instruction can 
be raised by individualizing these contingencies. 


SUMMARY 


This paper summarizes a general model used to 
design a computer-based instructional System. This 
idiographic contingency model defines c1 asses of 
variables that are presumed to be important in adap- 
tive instruction. It also specifies relationships be- 
tween those variables that characterize the learner 
before and after learning which can be used selec- 
tively to individualize instruction, Included amon, 
the “‘before’’ measures are knowledge, aptit wie 
and personality. The data indicate that the “before” 
measures have different relationships with the **af- 
ter" measures (e.g., amount learned and attitude) 
under different conditions of learning. These dat: 
support the model, а 


The kinds of decisions that 
individualize instruction were 
of research studies relating to 
briefly. Relevance of the data 
dicated and it was pointed out t] 
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TABLE 5 
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CORRELATIONS BETWEEN INTELLIGENCE SCORES AND PERFORMANCE VARIABLES - 


(а) MIDPROGRAM TEST, (b) POSTTEST AND 


(c) ERRORS IN PROGRAM - UNDER FOUR DIFFERENT 


CONDITIONS OF SOCIAL REINFORCEMENT (PARISI, 1965) 


16 P. F. Factor в? 
Intelligence with 


Verbal Reasoning 


with 
Group? LM LP E LM LP E 
Condition 1+ -.07 .33 -.22 -.15 .05 7.24 
Condition 2+ -.16 .51* -.17 .40 .31 DAE 
Condition 3- .38 .49* -.44 .40 49** E 
Condition 4 0 , 48* .42* -.46* .4T* 46* ^d 


The social reinforcement conditions 
reinforcement; Condition 2« is posit 
positive nor negative. 


are coded as follows: Condition 1+ mean positive 
ive only; Condition 3- is negative only; 


and negative social 
and Condition 4 o is neither 


bom is logic midprogram; LP is logic posttest; and E is errors. 


жр <.05 *#р<.01 


and (c) personality and evaluative feedback contin- 
gencies. 


The technological capability for accomplishing a 
high degree of individualization in instruction exists. 
The problem now is to find thecritical psychological 
and educational data which would make such a tech- 
nological capability an effective means of instruction. 


FOOTNOTE 


l. This paper appeared as Technical Report No. 12 
from the Training Research Laboratory, Of- 
fice of Naval Research, Contract NONR 3985 
iA Urbana, Illinois: University of Illinois, 
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"tmm Instructional Methods and Teacher Behaviors 


TEACHING METHODS, or the system of interactions which occur be- 
tween a teacher and students, 


the last decades. In this Part, t 
Presented. N. L. Gage points 
has not produced consistent, replicable or useable results. 
and more limited approach must be used, In 
search on Instructional Methods” 


the Paradigm set forth by Ned 


Traini A. Flanders in “Interaction Analysis and Inservice 
raining.” Flanders proposes a system of research in which a limited number of 
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ed at 3 second intervals, The resultant matrix of 
lysis of the verbal interchange between the teacher 
ysis is discussed in terms of ifs use as a research 
9 behavioral changes in teachers, such as inserv- 


— An Analytical Approach to 
Research on Instructional 


Methods 


INSTRUCTIONAL method constitutes 
one of the most important and promising but also 
the most frustrating of the areas of educational re- 
Search and development. Compared with the areas 
of learning, subject matter, instructional materials, 
and organization for instruction, instructional meth- 
od appeals to the classroom teacher as closest to 
the heart of her problem. Itis all right for a teach- 
er to know about learning, to know his subject mat- 
ter, to have appropriate instructional materials, 
and to fit into a given organization for instruction. 
But what a teacher really wants to know is, ‘‘What 
Should I do in the classroom?’ If you ask prospec- 
tive teachers or teachers оп the job, ‘Where do 
you really want help?" I think the reply will deal 
with some aspect of instructional methods. 


Unfortunately, the urgency of this demand has 
not been accompanied by corresponding success in 
meeting the demand. Research on teaching has 
yielded relatively few solid and usable results. 
The stature of theory and research in other areas 
puts them well ahead of the study of teaching in the 
Struggle for scientific maturity. 


Implicit in what I am saying is a basic distinc- 
tion between research on learning and research on 
teaching. The former deals with all the conditions 
under which learning, or a change in behavior due 
to experience, takes place. And as І have already 
indicated, the study of learning is relatively ma- 
ture, well established, with many volumes of sub- 
Stantial literature to its credit. Research on teach- 
ing, on the other hand, deals with a subset of the 
conditions under which learning occurs in one per- 
Son, namely, the conditions established by the 
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behaviors of another person, called the teacher. As 
our schools have developed during the past two or 
three millennia, we have always attempted to pro- 
mote and improve the learning process through the 
intermediation of such another person. Until very 
recently, the assumption that teachers were helpful 
or even necessary for many important kinds of learn- 
ing that society wanted to promote went unchallenged, 
Even today, the challenge of independent study or 
computer-assisted instruction and other devices is 
a mere whisper against the thunder of the assump- 
tion that teachers are necessary, that teachers are 
here to stay. My discussion of instructional methods 
is going to be based on that assumption. It is the rel- 
atively neglected, undernourished, and underachieving 
subset of research on learning which I call research 
on teaching that I shall deal with here. 


RESEARCH ON TEACHING 


У iables i 
Center's program. When the teachers' behaviors 


on teacher effects. On the other hand 


, 


and procedures serve as the independe 
then we have research on teacher educa: 
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both researchonteacher education andresearch on 
teacher effects as our domain, we have research 
on teaching, as itis understood intheStanford Cen- 
ter. My subject is the Center's program of re- 
Search and development on instructional methods " 
and my procedure will be to Work from the past to 
the present, from the general to the specific, and 
then to try fora look at where we are going. When 
Iget down to Specifics, I shall be dealing primarily 
with research in which Iam involved, rather than 
all of the research underway in our Center. 


PAST RESEARCH 


But first, let us look at where research on teach- 
ing has been. Asthe behavioral sciences go, it has 
а respectably long history but a regrettably inglori- 
ous one. Research on teaching has been going on 
almost as long as re search on learning. Some 
Studies were made in the 1910’s and 1920’s, and 
quite afew were made during the 1930's. By the 
early 1950's, substantial reviews and bibliogra- 
phies of research on teaching began to appear. And 
during the last decade, the flow of research on 
teaching has indeed become Significant. But the 
early years did not pay off in solid, replicable, 
meaningful results that had considerable theoreti- 
cal or practical value. Positive and significant re- 
sults were seldom forthcoming, and they survived 
replication even less often. The research yielded 
many findings that did not make sense, that did not 
hang together in any meaningful way. 


Under these conditions, as Kuhn (1962) has 
pointed out, research workers are impelled to re- 
examine their first principles, the paradigms by 
which they guide their efforts, The model prob- 
lems and solutions of the community of researchers 
on teaching were accordingly subjected to more 
and more reappraisal. Licking the wounds inflicted 


frameworks, approaches, and paradigms for re- 


ti 

ded pec: possessor be identified as a 
i wt "EA 
down in the Scale. Jod. ee 


is made quite specific, relating not to gener- 
al traits but to concrete procedure, the funda- 
mental difficulty remains, that there is no 
external and generally accepted criterion a- 
gainst which the scale can be validated to 
establish the significance of its items 
(Walker, 1935, pp. x-xi). 


This kind of writing implies that there is some 
magic variable that applies to all of teaching, for 
all pupils, at all grade levels, in all subject mat- 
ters, and in all objectives. The phrase ‘һе cri- 
terion of teacher effectiveness" betokens a degree 
of generality that has seldom been found in any Й 
branch of the behavioral sciences. It also запасі 
the mistaken notion that such a criterion, large - 
а matter of values, can be established on the bas 
of scientific method alone, 


RECENT RESEARCH 


The so-called criterion problem misled a whole 
generation of researchers on teaching, embroiled 
them in endless and fruitless controversy, and lured 
them into hopelessly ambitious attempts to predict 
teacher effectiveness over vast arrays and spans of 
Outcomes, teacher behaviors, time intervals, and 
pupil characteristics, all on the basis of predictive 
variables that had only the most tenuous theoretical 
justification in the first place. It is little wonder 
that, when Berelson and Steiner (1964) dealt with the 
subject of teachers' behaviors and characteristics 
in their inventory of scientific findings in the behay- 
ioral sciences, they dismissed the “large number 
of studies” with the single dismal sentence that 
“there are no clear conclusions” (p. 441). 


If the global criterion approach had proved to be 
Sterile, what was the alternative? The answer was 
to take the same path that more mature sciences 
had already followed: If variables at one level of 
phenomena do not exhibit lawfulness, break them 
down. Chemistry, physics, and biology had, ina 
Sense, made progress through making finer and 
finer analyses of the phenomena and events they 
dealt with. Perhaps research on teaching would 
reach firm ground if it followed the same route. 


Apparently, a number of students of the problem 
had this general idea at about the same time. In 
1962, writing my chapter on paradigms for the 
Handbook of Research on Teaching, I coined the 
term, ''micro-criteria" of effectiveness. As I 
Said in that chapter: 


++. One solution within the ‘‘criterion-of- 
effectiveness'' approach may be the develop- 
ment of the notion of **micro-effectiveness, '' 
Rather than seek criteria for the over-all 
effectiveness of teachers in the many, varied 
facets of their roles, we may have better suc- 
cess with criteria of effectiveness in small, 
specifically defined aspects of the role... 
a sufficient number of laws applying to rela- 
tively pure aspects of the teacher's role, if 
such laws could be developed, might even- 
tually be combined, ... to account for the 
actual behavior and effectiveness of teach- 
ers with pupils under genuine classroom con- 
ditions” (Gage, 1963, p. 120), 
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A group of workers at Stanford University, to 
which I was to move a few months later, took a sim- 
ilar view. In the Stanford program for training 
secondary-schoolteachers, Robert Bush, Dwight 
Allen, and their co-workers adopted what is now 
known as the technical skills approach. Technical 
skills are specific instructionaltechniques and pro- 
cedures that a teacher may use in the classroom. 
They represent an analysis of the teaching process 
into relatively discrete components that can be used 
in different combinations in the continuous flow of 
the teacher's performance. The specific set of 
technical skills adopted in the teacher-education 
program atStanford may bequite arguable. Indeed, 
the list of skills has been revised a number of times 
over the past few years. What is important is the 
approach – the attempt to analyze teaching into 
limited, well-defined, components that can be taught, 
practiced, evaluated, predicted, controlled, and 
understood ina way that has proven to bealtogether 
impossible for teaching viewed in the larger chunks 
that occur over a period of an hour, a day, a week, 
Or à year. 


When analyzed-teaching, in the form of techni- 
cal skills, is made the focus of our concern, we 
find it possible to do fairly satisfying research 
both on teacher education and on teacher effects. 
The satisfaction comes from being able to measure 
or manipulate relevant independent variables, per- 
form true experiments, or make careful analyses 
and measure relevant dependent variables. 


The idea of technical skills may be illustrated 
by the terms used in a recent list of such skills. 
One was called ‘‘establishing set,” or the estab- 
lishment of cognitive rapport between pupils and 
teacher to obtain immediate involvement in the les- 
son; one technique for inducing a positive set is the 
use of relevant analogies. A second technical skill 
is that called ‘‘establishing appropriate frames of 
reference,’’ or points of view. A third technical 
skill is that of ‘‘achieving closure," or pulling to- 
gether major points, linking old and new knowledge, 
at appropriate points within a teaching episode as 
wellas at the end. A fourth technical skill is that 
of ‘‘using questions" in such a way as to elicit the 
kinds of thought- processes and behaviors desired 
such as simple recall, or conce pt formation, or 
evaluation. Other technical skills are those in ‘‘rec- 
ognizing and obtaining attending behavior,’’ ‘‘con- 
trol of participation,” **providing feedback, '" ‘‘em- 
ploying rewards and punishments, '' and ‘‘setting a 
model. "' 


MICRO- TEACHING 


These technical skills into which important as- 
pects of the teaching job have been analyzed are 
not merely the subjects of lectures and discussions 
in the teacher education program. Rather, they 
form the basis for the intern's practice teaching 
prior to his entrance into actual classrooms. This 
Procedure, well known by now as '*micro-teaching,"" 
consists in getting the trainee to teach a scaled- 
down teaching exercise. Itis scaled down in terms 
of time because it lasts only 5 to 10 minutes. It is 
scaled down in terms of class size, because he 
teaches a group of not more than five pupils, who 
are brought in and paid to serve as pupils in the 


micro-teaching clinic. It is scaled down in terms 
of the task, since the trainee attempts to perform 
only one of the technical skills in any single micro- 
teaching session. The sessions are recorded on 
video tape, and the trainee gets tosee and hear him- 
self immediately after the session. While he looks 
at and listens to himself, he receives criticisms and 
suggestions from supervisors trained tobe both per- 
ceptive and tactful. Then he ‘‘re-teaches’’ the same 
lesson to a new small group of pupils in an attempt 
to improve on his first performance of the specific 
technical skill that is his concern in that session. 


Obviously, the general idea is subject to many 
variations. The size of the class can be manipu- 
lated; the number of trainees teaching a given group 
of children can be increased; the duration ої the les- 
sons can be lengthened; and the nature of the teach- 
ing task can be made more complex so as to em- 
brace a group of technical skills in their real-life 
combinations. But the idea of analyzing teaching 
into technicalskills remains the heart of the method 
and provides its power as a paradigm for research. 


The research on micro-teaching and technical 
skills in the Stanford teacher education program has 
taken the form of experiments in which various pro- 
cedures for feedback to the trainee are manipulated. 
Professors Dwight Allen and Frederick McDonald 
have organized a program of research on variables 
hypothesized to influence the learning of the techni- 
cal skills of teaching. Their independent variables 
fall into three categories: practice variables, feed- 
back variables, and demonstration variables. A 
practice variable may consist in micro-teaching 
versus teaching in an actual classroom, A feedback 
variable may be the positive or negative character 
of the feedback, or the mediation of the feedback by 
another person rather than the trainee himself. Fi- 
nally, a demonstration variable may take the form 
of symbolic demonstration, consisting of written or 
spoken words, or perceptual demonstration, consist- 
ing in either live or video-taped portrayals of the 
desired behavior; and each of these can consist ei- 
ther of self- modeling or modeling by others. Other 
independent variables have been identified, such as 
the timing of a reinforcement, the amount of prac- 
tice, and the amount of feedback. 


This condensed description of the Allen- McDonald 
research program can suffice to illustrate the use of 
the analytic approach to research on teacher educa- 
tion. Their research takes the form of true experi- 
ments in which subjects are randomly assigned to 
different values of the independent variable, 
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caused by volcanic eruptions. Everyday observa- 
tion tells us that some people explain aptly, getting 
to the heart of the matter with just the right termi- 
nology, examples, and organization of ideas. Other 
explainers, on the contrary, get us and themselves 
all mixed up, use terms beyond our level of com- 
prehension, draw inept analogies, and even employ 
concepts and principles that cannot be understood 
without an understanding of the very thing that is be- 
ing explained. To some of us, it has seemed that 
explaining comes very close to being the inner es- 
sence of instruction, so that when a teacher is at- 
tempting to explain proportionality to his geometry 
class or irony to his English class, he is behaving 
more purely asateacher than when he is attempting , 
Say, to motivate, promote discussion, or maintain 
discipline. At any rate, we have made some studies 
of explaining ability in the attempt to determine 
some of the characteristics of effective explanations. 


EXPLAINING ABILITY STUDY IN MICRO- 
TEACHING CLINIC 


The first study was made in the micro-teaching 
clinic at Stanford during the summer of 1965 by 
J.C. Fortune, R. F. Shutes, and N. L. Gage (1966). We 
attempted to determine the generality of explaining 
ability, that is, the degree to which the ability to 
explain one topic was correlated with the ability to 
explain another topic, and the degree to which the 
ability to explain a topic to one group of pupils on 
one occasion was correlated with the ability to ex- 
plain the same topic to another group of pupils on 
another occasion. We also were able to design the 
study so as to determine the degree to which there 
was generality over both pupils and topics, or the 
degree to which the ability to explain one topic to 
one group of pupils on one day correlated with the 
ability to explain another topic to another group of 
pupils on another day. Because there were only 
Sixty pupils to be shared in groups of five among 
approximately forty interns in the micro-teaching 

Clinic, the design became quite complex in order to 
avoid having any intern teach the same topic to the 


Same group of pupils more than once and to avoid 
having the same group of pupils receive an explana- 
tion of the same topic more than once. According- 
ly, the forty social studies interns — and we chose 
to work with the Social studies interns 

there were more 
tern - 
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the variance of the adjusted mean posttest compre- 
hension scores was attributable not to the ability of 
the pupils or the difficulty of the topic but rather to 
the differences among the teachers. We then inves- 
tigated the question of the various kinds of general- 
ity by determining the median intercorrelations a- 
mong the various means. The upshot of this part of 
the study was that generality over topics was non- 
existent, and generality over groups was about 4. 
In other words, the interns were moderately consist- 
ent in their ability to explain the same topic to dif- 
ferent groups on different occasions, but they were 
not consistent in their ability to explain different 
topics. 


The study dealt with the correlations PEN ou 
plaining effectiveness and the pupils udi e idet 
ious aspects of the explanations. The pupi i 
the interns’ performance with respect to twelve 
items, such as clarity of aims, organization of the 
lesson, selection of material, and clarity of Dresen- 
tation. To us it seemed that some of these dimen- 
sions should correlate more highly with explaining 
ability than others. In particular, we hoped that 
such discriminant validity would be manifest in the 
form of a higher correlationbetweenthe mean rating 
of the lecture for ‘‘clarity of presentation” than for 
any of the other items of the Stanford Teacher Com- 
petence Appraisal Guide. Our hope was borne out ; 
the correlation of the adjusted mean posttest com- 
prehension scores with pupils’ ratings of ‘‘clarity of 
presentation" was . 56, higher than that with any of 
the other rating scale items. This result seems to 
us to support the validity of both the index of lecture 
effectiveness and the mean ratings by the pupils. 


EXPLAINING ABILITY STUDY IN PUBLIC SCHOOLS 


During the school year 1965-1966, I was able, in 
collaboration with Barak Rosenshine, to undertake 
a replication and extension of this study in the public 
Schools (Rosenshine, 1968). Because there was no 
lack of students in the high-school classes, taught 
by their own teachers, we did not become involved in 
the complexities of design necessary in the vot ford 
teaching clinic. To put it very briefly, we B pd 
eleventh-grade social studies teachers le t'on 
liver a 15-minute lecture on an ‘‘Atlantic Repor The 
Yugoslavia taken from the Atlantic_Monthly. sin 
teachers had been given the article several By rat 
advance, and had been told to prepare а lecture i 
would enable their pupils to answer a 10- ШЕ eta 
ple-choice test of comprehension of the art lec- 
main ideas. To guide them in preparing Mo oies 
ture, they were given five of the multiplen e i five 
questions that would be asked, while the шше lec- 
questions were withheld. After the 15- Eu to use 
ture, in which the teachers were perm “ients took 
the blackboard but no other aids, their stu her’s lec- 
the 10- item test. They also rated the teac ch ° 
ture on'items similar to those I have already de 
scribed. Thenextday, the same teachers andclasses 
did the same things, except that the subject matter 
was an “Atlantic Report”? on Thailand; again the 
teachers had been given five of the ten items as à 
guide to the kind of lecture that they should prepare 
and had been told to focus on the explanation of the 
ene ideas, Concepts, and principles brought out in 
A article, which constituted the curriculum for this 
of teaching. On the third day, the classes heard 
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athird lecture, one that was the same for all 
classes, a tape recorded 15- minute lecture on Israel, 
a verbatim reading of an “Atlantic Report, ” and 
then the pupils again took a l0- item test based on 
that article. 


The class mean on the Israel test was used to 
adjust the class means on Yugoslavia and Thailand 
for between-class differences in ability. Our rea- 
Soning was that the score on such a test of compre- 
hension of a uniform lecture would be more useful 
in controlling relevant kinds of ability than would 
the usual scholastic aptitude test. Theclass means 
on Yugoslavia and Thailand were also adjusted for 
teacher differences in the content-relevance of the 
lecture, as determined by scoring the transcript of 
the lecture for relevance to the ten items on the 
comprehension test. We then assu med that the 
variance that still remained in the adjusted com- 
prehension test means of the classes would reflect 
differences between the teachers in what we were 
concerned with, namely, the intellectual style and 
process of theteacher'slecture. In this study, the 
teacher's adjusted effectiveness index on Yugosla- 
via correlated .47 with his effectiveness on Thailand; 
і, е. , there was considerable generality of effective- 
ness over topics, even after student ability and con- 
tent relevance had been partialed out. 


It should be noted that we were using the micro- 
teaching idea in this investigation. The teaching 
was restricted to just one aspect of the teacher's 
role, namely, ability to explain the current, social, 
political, and economic situation in another country. 
The curriculum was also scaled down. We also 
used another major feature of the micro-teaching 
clinic, the video tape recorders which made it 
possible for us to study the teacher's behavior, ver- 
bal and nonverbal, at leisure. 


One major question was that of whether our cri- 
terion, or micro-criterion, of teacher effectiveness 
in explaining, namely, the mean comprehension 
Score of the pupils, adjusted for both mean pupil 
ability and content-relevance, contained variance 
that would be manifested in something about the 
lecture that was visible or audible. In other words, 
was there some difference between good and poor 
explanations that was worth trying to analyze? So 
we picked two lectures on Yugoslavia that were ex- 
tremely high on our index of effectiveness and two 
that were extremely low. We had a group of eight 
judges read the article on Yugoslavia and take the 
comprehension test, and then watch and listen to 
all four of these lectures. Then the judges ranked 
the lectures in terms of perceived effectiveness in 
engendering comprehension as measured by the 10- 
item test. It turned out that the judges’ post-dic- 
tions were quite significantly more accurate than 
could have been expected on the basis of chance, 
and we were accordingly reassured that our micro- 
criterion was indeed reflected in something that 
could be seen or heard in the lecture. 


But the major concern of this investigation was 
to determine the cognitive and stylistic correlates 
of the lecture’s effectiveness, For this purpose,we 
used extreme groups to minimize the labor of scoring 
a host of variables about which we had no great 
Conviction. So the ten most effective explanations 


on Yugoslavia were identified and also the ten least 
effective. From these, we chose at random five of 
the most effective and five of the least effective. 
Then, groups of judges and content analysts worked 
over the transcripts of the lectures, scoring and 
rating them on a host of variables. Some of these 
were sentence fragments, the average sentence 
length, the number of prepositional phrases per sen- 
tence, andsoon. Other variables dealt with the 
number of self-references by the teacher, or with 
various aspects of syntax, or instructional set, fa- 
miliarization, uses of previous knowledge, mobilizing 
sets, attention focusing procedures, organization, 
emphasis, amount of repetition and redundancy, the 
number of words per minute, and so on. 


The variables that discriminated between the five 
best and the five worst lectures on Yugoslavia were 
then tried out on the other set of five best and five 
worst lectures on Yugoslavia to see if they still dis- 
criminated. Those that survived this first cross- 
validation were then tried out on the best and worst 
lectures on Thailand. At the last accounting, two 
characteristics of the lectures had survived this 
kind of validation and cross-validation procedure. 
These variables were what we are calling ‘‘explain- 
ing,” or the degree to which the teacher describes 
the how, why, or effect of something, and the ‘‘rule- 
eg-rule" pattern, or the degree to which the teacher 
states a generalization, gives examples of it, and 
then summarizes a series of illustrations ata higher 
level of generality than the illustrations themselves. 
These two variables not only seem to be valid in our 
data but also are reliably rated by independent judges. 
Nonetheless, these must not be considered to be 
firmly established findings; they are merely examples 
of the kinds of conclusions to which research of this 
kind can lead. 


Currently, we are in the process of Scoring all 
of the explanations on all of the variablesthat appear 
to hold any promise, and we will then undertake 
studiesof the complete correlation matrices involving 
not only the indices of explanation effectiveness, but 
also all of the characteristics of the explanations, 
and the ratings of the lectures by the pupils who 
heard them. Such a complete correlational study 
will throw light on the consistency from one lecture 
to another of the indices of lecture effectiveness, and 
the stylistic characteristicsof the lectures, and also 
their intercorrelations. 


, What I have been describing is of course a corre- 

lational study. Along with its advantages in permit- 

ting the exploration of a wide variety of possible cor- 
relates of explaining ability as they occur under 
fairly normal conditions, it also has the disadvantage 
of making causal interpretations hazardous 3 
this reason, studies of this kind ought to proc 
fairly rapidly into experiments in which the diff. 
ways of explaining will be based, at le 
leads obtained from our correlational 
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we can find in order to find out how they behave and 
what they are like so that we can attempt to produce 
more teachers like them. Many research workers 
See little promise in this approach. They recom- 
mend that we undertake instead to develop wholly 
new models of the teaching process designed for 
optimal effectiveness regardless of their similarity 
to the way teaching now goes on in the normalclass- 
room. 


COMPUTER- ASSISTED PROGRAMMED 
INSTRUCTION 


The teaching model that many advocates of this 
approach have in mind is that of programmed in- 
Struction, particularly computer-assisted pro- 
grammed instruction. As Suppes and Atkinson and 
others have described this revolutionary undertaking 
in research and development on instruction, it holds 
out great promise indeed. Before too long. the 
annoying problems in the hardware will have been 
solved. After a somewhat longer time, we may 
expect substantial and well-validated programmed 
curricular materials to have been developed in all 
the subject matters and grade levels. As one who 
has seen the highly developed installations at the 
Brentwood School in East Palo Alto, California, I 
must share the optimism of Suppes and Atkinson, 
and other developers of computer-assisted instruc- 
tion, 


Their very success. or coming success, raises 
problems for the kind of instructional methods with 
which we have been concerned in this paper. On 
superficial examination at least, certain major 
problems of ordinary classroom teaching seem to 
be clearly surmounted by computer-assisted instruc- 
tion. For example, the problem of the cognitive 
complexity in the teacher’s task, of how the teacher 
can say just the right thing at the right time to de- 
velop a concept or formulate a theory. is apparently 
well handled, at least in principle, by computer - 
assisted instruction. Its programs can be worked 
out and tried out in meticulous detail, well in ad- 
vance, at leisure, by the most skilled curriculum 
experts in the land, and then made available in all 
their subtlety and complexity to every teacher who 
uses the program. Another major problem in the 
ordinary classroom is that of individual izing instruc- 
tion. No matter how we group our pupils between 
schools, within schools, or within classrooms, we 
still have the problem of adjusting the rate and di- 
rection of the teaching and learning process to the 
needs and abilities of the individual pupil; here 
again, at least in principle, computer-assisted in- 


struction seems at first glance to have the better 
of the live teacher. 


i While pondering these problems, I got some 
elp from a restatement of the idea of individualized 


instruction in a recent paper by Philip Jackson (1966). 
He states it as follows: 


tora vidualizing instruction, in the educa- 
ense, means injectin i а 
lesson wh j g humor into 


5 еп а student seems to need it, and 
quickly becoming serious when he is ready to 


eem down to work; it means thinking of ex- 
съда uit are uniquely relevant to the 
кз 8 previous experience and offering 

em at just the right time; it means feeling 


ooo o oo ERR 


concerned over whether or not a student is 
progressing, and communicating that concern 
in a way that will be helpful; it means offering 
appropriate praise, not just because positive 
reinforcers strengthen response tendencies. 
but because the student's performance is de- 
serving of human admiration; it means, in 
Short. responding as an individual to an indi- 
vidual. 


Individualization in this sense is much more than 
allowing for differences in speed of moving through 
a program or providing different branches or routes 
through the material. 


Jackson's analysis of this kind of limitation in 
computer-assisted instruction should be placed 
alongside of the indications by Suppes (1966) that 
tutoring and dialog, which are higher levels of in- 
Struction than drill-practice, are still well in the 
future, as capabilities of computer-based instruction. 
Hence. any fears about the rapid obsolescence of 
live teachers, even where narrowly defined cognitive 
objectives are concerned, are quite unwarranted. 
That is, there will still be a need for teachers to 
use the kinds of technical skills, including explaining, 
with which the analytic approach being developed at 
Stanford and elsewhere is concerned. We shall have 
to continue to grapple with the problems of cognitive 
complexity and individualization through the medium 
of the live, human, teacher even in the realm of the 
well-formulated cognitive objectives. And there will 
always be the indispensable role of teachers inassist- 
ing pupils in attaining various kinds of affective and 
Social learnings in the classroom. 


Accordingly, group discussions, role playing, 
teaching for divergent thinking, as well as the tech- 
nical skills I have already mentioned, are all the 
subject of various research and development projects 
now under way in the Stanford Center for Research 
and Development in Teaching. We are also looking 
at the way in which the organizational context influ- 
ences the teacher’s choices among ways to teach. 
And, in one of our projects, entitled “Тһе Teacher 
in 1980,'’ we are looking at the way in which new 
curriculum developments, television and other tech- 
nical aids, computer technology, and new organiza- 


tional schemes inthe schools will affect the teacher’s 
role in the foreseeable future. 


CONCLUSION: A NEED FOR ANALYSIS 


In conclusion, let me refer again to what I see as 
one basic new the me in the research and develop- 
ment in teaching that is now under way at Stanford 
and elsewhere. If it were necessary to sum it up in 
one word, my word would be analysis, breaking down 
the complexities that have proven to be so unmanage- 
able when dealt with as a whole. We are no longer 
crippled by the notion that because there is one word 
called ‘‘teaching, ” there is one, Single, over-all 
criterion of effectiveness in teaching that will take 
essentially the same form wherever teaching occurs 
Even if none of the analyses of teaching that we have - 
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Instead, they will be replaced by other analyses of 
teaching, perhaps even finer analyses, until we get 
the sets of lawful relationships between variables 
that will mark the emergence of a scientific basis 
for the practice of teaching. It may well be that a 
15- minute explanation of a 5- page magazine article 
is still too large a unit of teaching behavior to yield 
valid, lawful knowledge. It may well be that the 
mean score on a 10-item test of comprehension, 
adjusted for student ability and content relevance 
of the lecture, is still too large and complex a de- 


pendent variable. But, compared with the massive, 


tangled, and unanalyzable units that have typically 
been studied in the past — in researchonthe lecture 
method, the discussion method, and class size, for 

example - such units seem precise and manageable 
indeed. And eventually, of course, we shall have 
to put teaching back together again into syntheses 
that are better than the teaching that goes on now. 
I think it would be safetosay that there is now some 
hope of our being able to develop a scientifically 

grounded set of answers to every teacher's central 
question, **What should I do in the classroom?" 
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Inservice Training 


WHAT IS INTERACTION ANALYSIS? 


INTERACTION analysis is a system 
for observing and coding the verbal interchange be- 
tween a teacher and his pupils. The assumption is 
made that teaching behavior and pupil responses are 
expressed primarily through the spoken word as a 
Series of verbal events which occur one after an- 
other. These events are identified, coded so as to 
preserve sequence, and tabulated systematically in 


order to represent a sam ple of the spontaneous 
teacher influence. 


The most important criterion which any coding 
system must meet before it can be considered sat- 
isfactory is that a trained person can decode the 
data in order to reconstruct those aspects of the 
original behavior which were encoded, even though 
he was not present at the observation. A part of 
this article will describe inferences which can be 
made from a blind analysis of coded data. 


Interaction analysis has been u: 
taneous teachin 


in projects wh 


sed to study spon- 
£ behavior and it has also been used 
F ich attempt to help teachers modify 
their behavior, In the first instance there may be 
a long period of time between observations and the 
analysis of the data, The data can be punched on 

M cards as they are collected over a period of 
Several months, buta Computer program to tabu- 
late and analyze the data may be used only after all 
the observations have been completed. Onthe other 
hand, when interaction analysis data are collected 


in order to provide a teacher with information about 


= Interaction Analysis and 
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his own behavior as a part of preservice or inserv- 
ice training, then it may be advantageous to code 
directly into adesired tabulation form so that inter- 
pretations canbe made at the earliest moment after 
the teaching episode iscompleted. This article will 
be more concerned with the procedures of interac- 
tion analysis which can be used during preservice 
and inservice training, and less concerned with 
applications in more basic research projects. 


A possible goalof an inservice training program 
might be to discover whether the spontaneous pat- 
terns of verbal communication which are observed 
are, or are not, consistent with the intentions of 
the teacher. In such a program, the assumption is 
made that modifying behavior in order to make it 
more consistent with intent will, in most instances, 
result in animprovement. The model toward which 
behavior is modified is created by the teacher at- 
tempting to change. 


An inservice training program can also be de- 
signed in which all participating teachers attempt 
to make similar modifications, for example, deyel- 
oping more skill in making full use of the ideas which 
are expressed by pupils during classroom discourse, 
Неге the model could be justified from the results of 
research which made use of interaction analysis, 
but special care would be necessar 
activities which would help teacher 
understand the desirability of such 
In brief, the value orientation wo 
ity of classroom instruction is j 
ideas expressed by pupils are 
ognized, clarified, used in so 


y in designing 
S accept and 
modifications. 
uld be that the qual- 
mproved when the 
more adequately rec- 
me step of a logical 
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analysis, thus giving the pupils a more active part 
in the learning activities. 


In nearly all applications of interaction analysis 
it is desirable to collect other kinds of data such as 
pupil attitudes, pupil achievement, and perceptions 
of the teaching situation held by the teacher and the 
pupils. These additional data permit the develop- 
ment of theory and explanation. 


The resources of such a program, as with all in- 
service activities, require money inorder to obtain 
time, space, and assistance for those who are par- 
ticipating. A number of steps can be anticipated: 
(a) Prospective observers and participating teachers 
must be trained to use the technique. This usually 
requires 6 to 12 hours under the direction of a quali- 
fied observer. (b) Personal, self-development 
goals must be clarified by the participating teachers. 
(c) Each teacher-observer team must set up com- 
parison situations involving a planned change of be- 
havior and two or more observations should be 
Scheduled so that evidence of change can be assessed. 
(d) Plans to collect other types of information must 
be completed. (e) Social-skill training sessions, 
based on interaction analysis categories, are often 
helpful, especially when they are incorporated in 
the observation training. Such training often helps 
teachers set personal goals for changing their be- 
havior. (f) Plans will be necessary to provide in- 
centives for teachers and to maintain the momen- 
tum of the program, once it is started. (g) Care 
must be taken to insure freedom from threat, to 
make sure that the voluntary aspects of participa- 
tion are genuine, and to avoid superimposing the 
program as an excessive demand on teachers who 
may already be too busy. 


OBSERVATION PROCEDURES AND 
MATRIX INTERPRETATION 


Given ten categories, shown in Table 1, all ver- 
bal statements are classified at least once every 3 
seconds by a trained observer. The events are coded 
by using the arabic numbers from one to ten 
which are written down in sucha way asto preserve 
the original sequence. The datacan then bé tabulated 
in a table of ten rows and ten columns which is 
called a matrix. 


Sucha series is entered into a matrix two at a 
time. The first number of each pair indicates the 
row of the matrix, the second the column. The 
first pair consists of the first two numbers. The 
second pair consists of the second and third numbers, 
and thus overlaps the first pair. All tallies enter 
the matrix as a series of overlapping pairs. 


With one tally approximately every 3 seconds, 
there are one hundred tallies for 5 minutes, 1, 200 
tallies per hour; therefore, 20 minutes, or apon 
four hundred tallies, provide a matrix with sufficient 
data for a number of inferences about verbal com- 
munication, 


In a sustained observation of a teacher covering 
Six to eight l-hour visits, it is necessary to tabu- 
late separate matrices for different types of class- 
room activities. Each matrix should represent 
either a single episode of class activity or any num- 


ber of homogeneous e pisodes that are combined. 
We use five activity categories for junior high school 
academic subjects; they are: routine procedures, 
discussion of new material, discussion to evaluate 
student performance or products of learning, general 
discussion, and the Supervision of seatwork or 
group activities. Different activity categories may 
be useful for a self-contained elementary-school 
classroom. In any case, the purpose of tabulating 
the data in several different matrices, instead of in 
just one total matrix, is determined by the purposes 
of observation and the range of expected classroom 
activities. 


In the case of the inservice training of teachers, 
data are tabulated into separate matrices in sucha 
Way as not to mask the comparison to be made. To 
illustrate, suppose the comparison is between two 
samples of teaching behavior, one before and one 
after several weeks of social skill training. Keep- 
ing the data in homogeneous activity matrices will 
help to avoid false conclusions. For example, a de- 
Crease, increase, or no change in certain categories, 
when the two matrices are compared, may be due to 
the differences in the learning activities observed, 
rather than due to change in teaching behavior which 
resulted from inservice training. Grouping the data 
to represent homogeneous learning activities helps 
to avoid such confusion. 


A tabulated matrix divides into special areas for 
interpretations that are shown in Table 2. Particu- 
lar questions can be answered by comparing tallies 
within and between these areas. Here are some ex- 
amples. 


Areas A (1+2+3+4), В(5+6+7), C(8+9), and D(10) 
can be used to find the percent time the teacher 
talks, the pupils talk, and time spent in pauses, si- 
lence, and confusion. Comparisons between Areas 
A and B provide information about the relative bal- 
ance between initiating and responding within teach- 
er talk. Initiating teacher talk is more directive, 
tends to support the use of teacher authority, and 
restricts pupil participation. Responsive teacher 
talk is more indirect, tends to share authority, and 
expands pupil participation. 


Area E is a block of nine cells that indicates the 
continued use of acceptance and praise, construc - 
tive reaction to pupil feeling, and clarifying, accept- 
ing, and developing pupil ideas, as well as transi- 
tions among these three categories while the teacher 
is talking. Inany inservice training program devoted 
to increasing the teacher’s attention to ideas ex- 
pressed by pupils, before and after comparisons 
would require an analysis of these nine cells, In 
fact, an inservice training program Which attempted 
to teach more subtle differences in the teacher's 
reaction to pupil ideas might require subdividing 
Category 3 in order to note the presence and ab- 
sence of various types of Category 3 statements 
For example, 3-1 -- merely re i 
the pupil ideas were heard; 3-2 -- reacts to specif- 
ic pupil ideas, but only in terms of the teacher’s 
perceptions of these ideas; 3-3 -- reacts to specifi 
pupil ideas, but reactions incorporate the E x 
tions of one or more pupils; and 3-4 -- eei 
reaction to a pupil's ideas by asking questions aa 
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TABLE 1 


CATEGORIES OF INTERACTION ANALYSIS 


Teacher Talk and Student Talk 


TEACHER TALK 


Response 


1.* Accepts feeling: accepts and clarifies the feeling tone of the students in a non-threatening manner. 


9.* 


Feelings may be positive or negative. Predicting or recalling feelings are included. 
Praises or encourages: praises or encourages student action or behavior. Jokes that release ten- 


sion, but at the expense of another individual; nodding head, or saying ‘‘um hm?" or ''go on” are 


included. 


3.* Accepts or uses ideas of students: clarifying, 
As teacher brings more of his own ideas into play, 


building, or developing ideas suggested by a student. 


shift to category five. 


4.* Asks questions: asking a question about content or procedure with the intent that a student answer. 


Initiation 


5.* Lecturing: giving facts or opinions about content or procedures; expressing his own ideas, asking 


rhetorical questions. 


6.* Giving directions: directions, commands, or order 
Т. ж Criticizing or justifying authority: statements inten 


s to which a student is expected to comply. 
ded to change student behavior from non-accept- 


able to acceptable pattern; bawling someone out; stating why the teacher is doing what he is doing; 


extreme self-reference. 
STUDENT TALK 


Response 


8.* Student talk — response: talk by students in response to teacher. Teacher initiates the contact or 


solicits student statement. 


Initiation 


9.* Student talk - initiation: talk by students which they initiate. 


If ‘‘calling оп’ student is only to in- 


dicate who may talk next, observer must decide whether student wanted to talk. If he did, use this 


category. 
10.* Silence or confusion: 


pauses, short periods of silence and periods of confusion in which communi- 
cation cannot be understood by the observer. 


*There is NO scale implied by these numbers. 
of communication event. 
position on a scale. 


that other pupils react. In effect, Category 3 is ex- 
panded into four categories for a special purpose. 


This would result in a 13 x l3 matrix instead of a 
10 x 10 matrix. 


Area F is a block of four cells that indicates the 
continued use of directions and criticism and transi- 
tionsbetweenthesetwocategories. The two transi- 
tion cells are particularly reliable indicators of dis- 
cipline problems. Shifting from directions to criti- 
cism is tallied in the 6-7 cell, and indicates that 
ee compliance is judged unsatisfactory by the 

cher. Shifting from criticism back to directions, 


the 7-6 cell indi i 
cates ar i 
after criticis : eturn to more directions 


Areas О and G9 ar i 
e i 1 a 
cause they isolate t ракш Hitere&ting bo 


he immediate 
б response of the 
xd a the moment students stop talking. One 
CH a ior еы can be discovered by 
e ance of indirect and direct state- 
ments shown їп G} and G with those found in Arean 


AREE 


Each number is classificatory; it designates a particular kind 
To write these numbers down during observation is to enumerate, not to judge a 


А апа B. The difference between superficial, short, 
perfunctory praise or clarification, and praise or 
clarification that is more carefully developed is 
easily seen by comparing thetallies in Area Сү with 
those in E, particularly the 2-2 and 3-3 cells. 


Area H indicates the types of teacher state- 
ments that trigger student participation. Respons- 
esto the teacher are found in column 8; state- 
ments initiated by the student in column 9. As one 
might expect, there is usually a heavy loading of 
tallies in the 4-8 cell. High frequencies in this 
cell and the 8-4 cell, but not in the 8-8 cell, often 
indicate rapid drill. 


Area I indicates sustained student participation. 
These may be lengthy statements by a few students, 
or student-to student communication. 


So-called ‘‘steady state’’ cells fall on the diago- 
nal from cell l-1 to 10-10. Tallies here indicate 


è 
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TABLE 2 


AREAS OF MATRIX ANALYSIS 


CLASSIFICA- cate- | 1 


CATEGORY TION {GORY 


4 5 6 | Fa 8 9 10 |/TOTAL 


ACCEPTS FEELING 


STUDENT IDEA 


PRAISE RESPONSE 2 Area E 


ASKS QUESTIONS 


LECTURES 


GIVES DIRECTIONS 


CRITICISM 


STUDENT RESPONSE 


STUDENT INITIATION 


КАС | -----— 


Area G, 


io SILENCE 


that the speaker persists in a particular communi- 
cation category for longer than 3 seconds. А11 
other cells are transition cells moving from one 
category to another. 


Outlined in Ње center of Table 2 by dash lines is 
the content cross. The total number of tallies in 
this area, compared with tallies not in this area, 
gives a very crude indication of the content orienta- 
tion of the class activity. 


In addition to making use of the areas just des- 
cribed, the following procedure can be followedto 
interpret a matrix. 


First, locate the single cell within the ten rows 
and columns which has the highest frequency. The 
Pair of events, represented by the cell, is the most 
frequently occurring and can be used as a starting 
Point in reconstructing the interaction. 


aui oni » from this highest frequency cell, you 
to be m aud Or backward, in terms of sequence, 
indicates ae diagram. The row of any cell 
event which с, 1081 likely third event, that is, the 
nal pair of 15 most likely to follow, given an origi- 

events designed by the highest frequency 


Area B 


Initiation 


cell. The column, onthe other hand, indicates 

which event most probably preceeded the pair of 
events in question. The flow of events is properly 
represented when the eye scans the matrix in a 

clockwise rotation. Should the highest frequency 
fall into a transition cell, not a steady state cell 
the row or column of either number in the pair can 


be studied to retrace or advance the se ue 
seq 
| nce of 


An example of matrix interpretation wi 

Shown later in this article. Su in tux ы 
pretation, however, is not likely to develop fro 
reading this article, which serves only to pro эы 
guidelines. For that matter, skill in Observ. e Э 
cannot develop from reading about how it is don 
All aspects of interaction analysis require practice 
in order to develop skill. It is the opposite of ae 
spectator sport. а 


FEEDBACK AND CONSULTATION 
WITH TEACHERS 


The purpose of feedback is to provide а 
with information about his verbal Saison 
permits a comparison with some standard or m р 
of what should have happened. Without pornos. 


A 
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comparisons which are planned in advance of the ob- 
servation, the reactions of the person receiving the 
information may be reduced to incidental specula- 
tions or points of interest that happen to be noticed. 


To be useful as a model or standard, one's inten- 
tions must be specified in termsof frequenciesto be 
found in the cells of the matrix. Thus, if a teacher 
wishes to practice providing more extended praise, 
he will expect to find an increase in the 2-2 cell of 
the matrix, one situation compared to another. If a 
teacher decides he would like to stimulate more pu- 
piltalk in which ideas are initiated by pupils, he 
might study column 9 inthe matrix to see what events 
trigger these pupil statements. In a program of in- 
service training in which all teachers attempt to in- 
crease the utilization of ideas expressed by pupils, 


attention will be directed to column 3 and row 3 of 
the matrix. 


Most inservice training programs can achieve 
some success in bringing selected concepts and value 
orientations to the attention of teachers, these are 
matters ої awareness. Evidence that such awareness 
has been implemented through overt behavior re- 


quires an objective assessment of spontaneous teach- 
ing behavior. 


The assessment of the spontaneous behavior must 
be reasonably objective in order to be reliable. Un- 
fortunately, interaction analysis is not free of bias 
and error, probably about one out of every ten classi- 
fications of an experienced observer is incorrect. 
Interaction analysis data can be and probably are 
more objective, when dealt with in summary form, 
compared with most other procedures for making 
judgments about spontaneous teaching behavior. 
Judgments about events which occur within time 
segments of only a few seconds and which must be 
repeated again and again tend to become more consist- 
ent with practice. Furthermore, noting the presence 
or absence of a short event is not a procedure which 
lends itself as easily to distortion and bias. 


No matter how objective, reliable, and valid an 
assessment procedure, the results will be distorted 
if the behavior itself is distorted. Unfortunately, 
merely anticipating observation might cause non-re- 
presentative behavior to appear, not to mention the 
observation experience itself. Below are some pol- 
icies and suggestions which we have found helpful 


in reducing the tendency of a teacher to put on an 
act while being observed, 


2866 ап Observer should be in the classroom 
only when invited by the teacher. 


с Observa- 
information which is relevant 
question which is considered 
articipants. Thus, a teacher 


to react to Pupil ideas. One les- 
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son would be observed before training and the sec- 
ond lesson after training. The plan could be em- 
bellished to provide greater insights by collecting 
additional data. For example. predictions about 
pupil perceptions, teacher perceptions, pupil atti- 
tudes and similar phenomena could be made, one 
lesson compared with the other. Then, instead of 
merely counting the incidence of constructive teac her 
reactions to ideas expressed by pupils, certain 
theories about the consequences of such teacher be- 
havior might be investigated. 


Third, the status and power difference between 
the observer and the teacher should be a minimum. 
Another teacher who is a best friend might make 
the most appropriate observer, providing skill in 
Observation is present. 


Fourth, the conference to provide feedback should 
follow a logical plan of inquiry. А11 the relevant 
data should be at hand and referred to in terms of 
questions to be answered and not in terms of idle 
curiosity. It often helps to have two or more ma- 
trices, since this facilitates the making of compari- 
sons out of which theoretical explanation grows. A 
single matrix more often stimulates opinions about 
what is **good"' and ‘‘bad,’’ illustrated by the ques- 
tion, ‘‘Do you think the lesson was satisfactory ?" 
Such questions place the observer in an awkward 
position, since he must express a general judgement, 
In this position he cannot be an equal partner in the 
inquiry process. On theother hand, when a hypothe- 
sis about behavior is being investigated and profes- 
sional competence of the teacher is not in the fore- 
ground, the analysisof data is more systematic and 


unwanted defensive distractions are less likely to 
occur. 


Fifth, the entire procedure including planning, 
execution, and analysis usually works more smooth- 
ly when the teacher, as well as the observer, has 
had approximately equal experience in observation. 
For example, setting up hypotheses, designing two 
comparable lesson plans, and knowing where to look 
in a matrix for the proper information are phases 
of the experience which should be shared by two 
partners who are equally competent. When the ob- 
server is more experienced and competent, the 
teacher defers and becomes the dependent member 
of the team. Both members of the team should 


have had previous experience in both teaching and 
observation. 


ILLUSTRATIONS OF 
PRACTICAL PROCEDURES 


For purposes of illustration, let us assume that 
the goal of inservice training is to increase the 
teacher's skill in making use of ideas expressed by 
pupils. In an article as short as this one, only four 
aspects of such an inservice training program will 
be mentioned. First, some initial performance data 
provides a before training performance pattern for 
which subsequent observation data can be compared. 
Second, skill training procedures canbe closely cor^ 
related to observation procedures. Third, data in 


J 
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Tante d tallies in categories 1+ 2+ 3 + 4 by 5 + 6+ Т to ob- 
tain this ratio). The highest cell frequencies are 
OBSERVATION MATRIX found in the steady state diagonal cells, such as the 


5-5, the 8-8, and the 9-9 cells. This suggests that 

| j the teacher and pupils were able to continue a parti- 
*- aa ada cular mode of expression once it started. Higher 
frequencies in these steady state cells indicate that 
the tempoof exchange was slower, for example, than 
might occur in a drill period. All silences were 3 
seconds or less (note no tallies in the 10-10 cell) and 
most pauses followed teacher questions (N = 5 in the 
4-10 cell) and teacher lecture (N - 3 in the 5-10 cell) 
rather than pupil statements. In six of these eight 
transitions, it was the teacher who broke the pause 
by talking (note that N - 1 in the 10-8 and 10-9 cells). 
This analysis of silence supports the interpretation 
that the teacher tended to initiate and did not permit 
a pupil more than 3 seconds to respond. 


The matrix is sometimes more easily understood 
when itis translated into a flow pattern illustrated in 
Figure 1. In this diagram the most frequently occur- 
ring steady state célls are shown as rectangles and 
the size of the rectangle indicates the relative fre- 
quency of the pair. Transitions among these cells 
are indicated by arrows and the thickness of the 
arrow is roughly proportional to the frequency of 
these transitions. Anyone can learn to draw such 
a pattern flow diagram from a tabulated matrix. Be- 
gin withthe highest frequency cell, in this case the 
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addition to interaction analysis are helpful in decid- 
ing whether or not a change in behavior is an im- 
provement. And fourth, more advanced training de- 
signs and more complex data collection procedures 
will be necessary to push progress beyond the ini- 
tial results. Each of these four topics will be dis- 
cussed in turn. 


Tolk—Response 


8-8 


N=23 


INITIAL PERFORMANCE DATA 


Initial performance data might be in the form of 
a short observationduringa class discussion. Table 
3 shows a matrix of a teacher which will now be in- 
ter preted to show how the observer and teacher make 
an initial diagnosis before training. The same data, 
of course, can serve as a before training standard 
in order to determine whether change has occurred. 


Sustained Pupil 
То —Iniiation 


9-9 
N-32 


] 
! 
Since the total tallies equals 380, one can esti- Nip ! 
mate that the matrix represents about 19 minutes of ! 
interaction (100 tallies = 5 minutes, 20 tallies = 1 ! 
minute), | 
I 
A number of percentages and other ratios, which ff 1 
help to form an initial picture, can be found at the 4 
bottom rows of Table 3. For example, the teacher Ou T ws] : 
talked 58.7 percent, the pupils 39.2 percent, and oF Pep is | 
Silence and confusion was 2. 1 percent. The teacher | 
was fairly directive, that is he initiated more than _ 
һе responded, as shown by an I/D = 0. 77 (divide all Figure 1z Initial Flow Pattern Before Training 
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5-5 cell with eighty tallies. Proceed across the 
same row to find out the next most likely Occurring 
events and inspect columns to identify the most 
likely preceding events. The arrows in Table 3 il- 
lustrate the clockwise direction of the flow. Thus 
in both the matrix and in Figure 1, begin with the 
5-5 cell, sustained lecturing. The next most likely 
event is that the teacher will ask a question. Next, 
pupils are most likely to respond to this question, 
See category 8. There are two events of almost 
equal probability following pupi] response talk, one 
is thatthe teacher will ask another question and the 
otheristhatthe teacher will lecture. The four tran- 
sitions shown by the heavy arrows in Figure 1 ac- 
count for most of the interaction: these are, 5-4, 
4-8, 8-4, and 8-5. Other transitions do occur, but 
the transitions just listed and their associated steady 
State cells account for 285 tallies or about 75 per- 
cent of the entire observation. 


Many things could be said about this pattern of 
interaction, but by far the most certain inference 
is that in the event that this teacher increased his 
use of category 3, it could be clearly seen in the 
matrix, The use of category 3, especially the 3-3 
cell, is below average for a classroom teacher. 
After training along these lines a second matrix and 
flow pattern diagram could be made as a compari- 
son with this initial performance. 


SPECIFIC SOCIAL SKILL TRAINING EXERCISES 


Specific social skill training exercises can be 
conducted which are closely correlated with inter- 
action analysis. A training skill exercise might in- 
volve three or four people who take turns providing 
certain patterns of verbal statements while respond- 
ing spontaneously to the statements of others in 
the group. The person being trained is designated 
as the actor, one other person is the observer, and 
the rest are reactors or foils who provide spontane- 
ity. Different assignments are given to the actor 
which he then carries out by interacting with his 
foils. The observer keeps a record of the communi- 
cation bv, coding within the ten categories. The 
assignment can be given in terms of the categories, 
for example, the actor is asked to produce a 4-8-4- 
8-4-8-4-8, ete, » Sequence. Anactor might be asked 
to choose atopic and produce any sequence which is 


relevant to teaching and is related tothe goals of the 
training. 


The activity of a training exercise follows a mod- 
el in which an actor attempts a particular pattern, 
his foils provide a Spontaneous setting, and the ob- 
Server keeps a coded recordof what happens. Some 
training episodes may last only afew minutes, others 
are longer, Each episode is followed by a discussion 
9f the performance and ways that it canbe improved. 


While this kind of activity can easily degenerate into 
Superficial pl; 


TUE ‘ay acting, especially when first attempt- 
ed, it is also true that serious and interested par- 
ticipants сап use this technique to practice producing 
Certain patterns of behavior under increasing diffi- 


cult circumstances and m: i 
ake progress in the self 
control of Spontaneous behavior, 


„чаш how to make full use of the ideas ex- 
pressed by pupils is a goal to which spontaneous 
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Skill training exercises can easily be adapted. It is 
possible to gain practice in listening carefully, in 
listing ideas expressed by another person, in sum- 
marizing ideas which have just been expressed, in 
building questions on ideas expressed by others, and 
to teach pupils to initiate their own steps in problem 
solving by showing a pupil how his ideas are related 
to a problem. 


Skill practice sessions become more realistic 
when they are closely related to classroom practice 
Sessions. After one or two initial observations an 
observer and teacher may discuss the relationships 
between the teacher's intention and the patterns 
which appear on several matrices, Skill practice 
sessions are then designed to emphasize a personal 
goal of the teacher based on discrepancies revealed 
by the matrix. If the training activities are custom 
built to the needs of the teacher, and if he sees them 
as relevant to his own professional development , 
then he is more likely to approach the training ses- 
Sions with serious intent and a sense of optimism 
concerning personal development. It also helps 
when classroom patterns begin to change in a de- 
sired direction as a result of skill practice sessions. 


Usually, the assignments of actor, foil, and ob- 
Server are rotated during skill practice sessions. 
Inthis way a teacher not only becomes familiar with 
the coding system of interaction analysis, he also 
obtains some degree of observer proficiency. He 


also learns to accept and developthe perceptions that 
others have of his behavior. 


The primary purpose of these training episodes 
is to practice producing certain patterns of state- 
ments, totranslate conceptsabout teaching into spon- 
taneous behavior patterns, to learn how to assess 
Such patterns, and to discuss how these various pat- 
terns appear in classroom teaching. Many varia- 
tions of training exercises can be directed toward 
these goals making them useful for inservice training. 


DECIDING WHEN A CHANGE IS AN IMPROVEMENT 


Deciding when a change is an improvement usual- 
ly requires data in addition to interaction analysis, 
For example, merely giving a teacher instructions 
to "go in there and produce more threes,” much 
like a football Coach, is quite likely to increase the 
incidence of category three. This change may take 
place, however, without any insight into the teaching 
process and may possibly be seen as either inappro- 
priate or not realistic by the pupils. 

Utilizing the ideas expressed by pupils during 
classroom discourse involves several pedagogically 
Sound principles. One such principle is that this 
kind of teacher behavi he perception 
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Another pedagogical principle is that a pupil 
learns to cluster ideas because they are sim- 
ilar and then abstract the cluster with an appropri- 
ate labelthroughdirected practice in expressing his 
ideas. A second observer can keep an inventory of 
separate ideas expressed by pupils, he can note in- 
stances in which the pupils, instead of the teacher, 
noticed a cluster, and finally, he can record wheth- 
er the teacher or the pupils supplied an appropri- 
ate label to a given cluster. This additional evi- 
dence also helps to show whether a change in teach- 
er behavior is, or is not, an improvement. 


PUSHING TRAINING BEYOND SUPERFICIAL 
CHANGE 


Pushing training beyond superficial Change usu- 
ally requires special category systems of inter- 
action analysis. For example, the division of cate- 
gory 3, mentioned earlier, can provide a record of 
different kinds of teacher reactions to the ideas ex- 
pressed by pupils. A similar expansion of catego- 
ries 8 and 9 will show variations in the different 
kinds of pupil statements. Often the expansion of 
teacher talk categories can provide more intricate 
and difficult social skilltraining assignments. 
Working back and forth between more elaborate 
category systems and more complex spontaneous 
social skill training assignments helps a teacher 
understand principles of pedagogy in terms of his 
own behavior. Unless a teacher can act out his in- 
Sights about teaching, these insights are of little 
use for the improvement of instruction. 


SUMMARY 


The use of interaction analysis in an inservice 
training program places an emphasis on the analy- 
sis of spontaneous verbal behavior. This emphasis 
helps to translate ideas about teaching into class- 
room application. 


For those who would like to try classifying ver- 
bal statements, but are inexperienced, a number of 
references are listed at the endof thisarticle which 
include suggestions for beginning. 
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Wn The Current Scene 


THE CURRENT scene in educational research and development is one 
of rapid and unprecedented change brought about largely by the advent of Fed- 
eral funding. In the first paper in Part V, R. Louis Bright and Hendrik D. Gideonse 
trace the development of these changes with reference to involvement of the 
Federal government starting with the 1954 legislation. They cite the national need 
for the results of educational research and development as the basis for the de- 
cision of the government to provide grant dollars for developmental projects and 
the design of the necessary strategies to implement programs of research and 
development. The evolving strategies of research and development designed to 
improve education are delineated by the authors and include 1) curriculum devel- 


opment, 2) research and development centers, 3) the educational research infor- 


mation centers, 4) а program for training educational researchers, and 5) a sys- 
tem of autonomous educational laboratories, These strategies have developed rap- 
idly and promise to provide the basis for much of the future growth of educa- 
tional research and development. 

The costs of educational research and development have grown from 
the original 1.7 million appropriated in 1957 to the present level of approxi- 
mately 200 million. This represents less than one-half of one percent of the total 
amount expended for education by all phases of our society, a figure which is far 
under the standards for research and development investments in most major in- 
dustries, Clearly additional growth can be anticipated. The authors of this paper 
Propose additional major expenditures in research and development amounting 
fo some billion dollars annually which would represent the still modest figure of 
Approximately 2.2 Percent of the total expenditures for education. 

. This continued growth of research and development in education, 
Which is almost exponential, calls for careful and systematic planning. In an. 
€ Bureau of Research, U. S. Office of Education, has pro- 
Ucational Policy Centers to conduct studies for estab- 
rections of educational research and development 5, 10, 


pu» 


as 


20, and 30 years hence. The rationale of these policy centers is discussed by Bright 
and Gideonse. 


The rationale and focus of a specific Research and Development Cen- 
ter is presented by Herbert J. Klausmeier, The Director of the Wisconsin Research 
and Development Center for Cognitive Learning. The Research and Development 
Center concept was proposed by the U. S. Office of Education in an effort to better 
coordinate research and development activities and to establish programatic re- 
search and development in specific areas related to the educational process. The 
Wisconsin Center is described in terms of eight components of operation: (1) prob- 
lem area, (2) outcomes sought, (3) research and development program and 
strategy, (4) dissemination and adoption program and strategy, (5) staff, (6) 
administration and management, (7) locus of operation, and (8) source of sta- 
bility of funding. Klausmeier points out that the programatic activities of the Wis- 
consin Center which have been brought sharply into focus now include 1) proc- 
esses and conditions of learning, 2) processes and programs of instruction, 3) 
facilitated environments. Other programs deal with dissemination, training activi- 
ties, and technical support. Such a center concept brings to bear on problems of 
education the diverse talents of the interdisciplinary staff available at a major 
American university. 


Most new and complex activities are more readily understood if 
represented in a model idealizing their essential components and operations. The 
total research and development effort in education is no exception. The model 
proposed by Hendrik Gideonse in the final paper recognizes the independent 
characteristics of the research act and the development act. In a prior model 
Proposed by Guba and Clark the research activity precedes and feeds results 
into the development activity which in turn provides materials and programs for 
adoption. However, Gideonse proposes a model which emphasizes the nonline- 
arity of the research and development complex. He states, "My purpose in de- 
veloping an alternative model is to create a heuristic which illustrates the essen- 
tial differences between research and development activities and show how the 
two are—or can be—related to one another and to the operating educational 
system." 

The model proposed emphasizes the quite different objectives and 
outputs of the research and development activities. Research is characterized as 
an activity begun with specific outcomes not being known. Contrasted to this is 
the development activity which is begun with the objective known or established 
at the onset. In the model proposed the dynamics of initiation of research and 
development activities are quite different, as are the outcomes or products, Re- 
search may be initiated for its own sake and pursued solely for the knowledge 
which it produces. Whereas development activities are initiated in response to a 
specific need; the specifications for the product can frequently be defined before 


the onset of the development activity. However, even though the utilization of 
the research product is not predicted or predictable there rests on the researcher 
the obligation to present results in such a way so as to allow transfer to other 
research or development projects. In the model proposed by Gideonse the out- 
put function of the various activities is crucial. Such a model allows those con- 
cerned with planning research and development activities to more clearly see the 
essential characteristics of these activities. 

The authors of this section describe the current scene and provide a 
glimpse of the future of educational research and development. The growth of 


the present research and development investment may indeed be a prologue to 
the future. 


> 
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=== Research and Development 
Strategies: The Current Scene 


IT IS MOST stimulating to partici- 
pate in a field which has generated as much excite- 
ment as educational research has in recent months. 
Controversy, discussion, and debate are very much 
the order of the day; the gradual growth of contacts 
with new and informative sources of data, and of 
faith in the research effort are developments full of 
promise for the future. 


In agreeing to prepare this paper, we explicitly 
accept the notion that it makes sense to consider 
Strategies of research and development for improving 
education. At the same time we do not make any 
claim to producing anything more than tentative 
steps in the direction of more adequately defined 
strategies for achieving mutually agreed upon objec- 
tives. For a variety of reasons which we will at- 
tempt to spell out in this paper, we have begun the 
process of developing and applying more explicit 
strategies to the research and development process. 
These strategies are neither rigid nor lightly held. 
Some notions of what we are about and why are 
clearly necessary; not to have them is to default a 
major responsibility. Inthis paper we try to Spell 
out both the general outlines of the strategies we 
are pursuing and the procedures by which we hope 
to continually modify the specific substance implicit 
inthose Strategies and the actual strategies them- 
selves, 


The improvement of American education depends 
upon the systematic investigation of the process and 
the necessary conditions for learning, the develop- 
ment of instructional objectives, strategies, and 
materials based on the knowledge educators and 
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others accumulate about the learning process, and 
finally on the implementation of those strategies and 
the use of those materials in instructional settings 
across the country, We take this as a fundamental 
article of faith, but it is a faith which has been sub- 
stantiated over and over again in all fields of human 
endeavor whether they relate to Space, health, poverty, 
industry or education. To claim, however, that sig- 
nificant and continuing advances in the process of 
education depend upon research and development is 
only the barest of opening statements, A perspective 
and the setting need first to be sketched in, and then 
a full exploration of the ramifications of that article 
of faith needs to be conducted. 


SETTING THE PERSPECTIVE 


Any consideration of the role of research and de- 
velopment in American education necessarily relates 
to the nature of the existing ‘‘system"” through which 
we have institutionalized education in our society 
With legal responsibility for education inthis count: ; 
vested inthe states and operational responsibility id 
Schools largely delegated to some 25,000 local edu- 
cational agencies authorized by the states, the rol 
of the Federal government has largely been confined 
to providing specified categorical financial Support 


These different levels of responsibili 
i i Š 
plicated by another feature of [тазны m 
A pony of the diverse institutions and 
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education, we are being generous. What we have is 
a de facto system which serves the broad needs of 
the society well, but perhaps not as well as it might 
if it were generously laced with efficient communi- 
cation channels designed to make it an integrated, 
functionally related entity of many parts and many 
purposes. 


From the point of view of the general improve- 
ment of education, it makes more sense to talk of 
the educational system as composed of classes of 
institutions and agencies. Each class embraces 
organizations which perform similar functions and 
which, if there were an organized System, would 
be expected to fulfill a particular role. Thus, the 
classes of institutions include the three levels of 
responsibility for education - local, state, and 
federal. They include professional associations 
and institutions for training teachers. They include 
the industries which supply materials and equip- 
ment to the schools. They include universities and 
institutes under whose aegis research on education 
and learning is performed. The existing ‘‘system,”’ 
in other words, is largely a collection of nearly un- 
related sub-systems or institutions representative 
of sub-systems. This feature is of no small im- 
portance in devising strategies for the general im- 
provement of the whole. 


A NATIONWIDE NEED FOR EDUCATIONAL 
RESEARCH AND DEVELOPMENT 


The nationwide nature of the need for educa- 
tional research and development is becoming in- 
creasingly apparent. Schools, districts, state 
education agencies, private institutions, colleges, 
and universities all have major needs relating to 
the improved fulfillment of their instructional re- 
sponsibilities, While individual institutions and a- 
gencies differ, many if not a majority of the prob- 
lems which confront them are similar, Serious 
questions of economics of scale have been answered 
in effect by the relatively small amounts of money 
Which have been devoted to educational research at 
the local level. We have seen for decades an entire- 
ly reasonable response on the part of local and state 
agencies with respect to the funding of major re- 
search and development efforts, While they have no 
doubt been aware that Sophisticated, well-conceived 
research programs would benefit not only them- 
selves but also many others as well, very few have 
felt that they theniselves could assume the financial 
and human burden of mounting large-scale research 
programs. It would make vastly more sense if 
the actual cost of generating those findings and 
materials were amortized more proportion- 
ately across all the potential beneficiar- 
ies. 


The method for teaching mathematics, for example, 
is not likely to be much different for similar chil- 
dren in Maine and Arizona. On the other hand, there 
may be significant differences in instruction in 
mathematics within a state for children having dif- 
ferent socio-economic characteristics, learning 
styles, or motivational problems. It would be a 
philanthropic district or state that felt it could mount 
the requisite research and development projects to 
serve all the immediate local needs when some other 
mechanisms could be develo ped which would 


distribute the cost more fairly in return for the mu- 
tual benefits that would be derived. 


THE EMPLOYMENT OF NATIONAL RESOURCES 


The resources for conducting educational research 
and development, though not as limited as some 
people would have us believe, are in fact unequally 
distributed across the country. No necessary con- 
clusions with respect to continuing patterns of sup- 
porting educational research can be drawn from that 
fact, but a consideration of it is certainly one of the 
relevant contextual matters for the development of 
research and development strategies. The nation- 
wide character of the need for research and the un- 
equal distribution of the human resources for re- 
search reinforce one another. They highlight the 
breadth of the problem, In other words, so long as 
human resources are unequally distributed, the di- 
rect or implicit coordination of them to serve re- 
search and educational needs of high priority will 
constitute one of the major background considerations 
in research strategies. 


FINANCIAL RESOURCES 


Certainly one of the more important matters 
lending perspective to research and development 
(R and D) strategies for improving education is the 
availability of financing. The resources are limited, 
Money available from all sources for R and D in 
education has never exceeded | percent of the total 
annual expenditures on education in the nation asa 
whole. Estimated expenditures for this fiscal year 
for all levels of formal education are $48. 8 billion.: 
By contrast the funds available at the Federal level 
for educational research and development are some- 
where between $200- 220 million. ? Simple arithme- 
tic reveals that this amount constitutes less than half 
of 1 percent of the total annual expenditure on educa- 
tion. We do not by any means take that as ‘‘a given’ 
for the future, but it is most assuredly an important 
element in developing perspective on the problem of 
research strategies for the present. 


THE DISTINCTION BETWEEN "RESEARCH'' AND 
“IMPROVEMENT” 


A final element for consideration in this context 
is our conviction that R and D in education and 
the change process in education are two very different 
functions or processes which must, however, be 
related intimately, sensitively, and integrally to one 
another. The research and development process pro- 
duces the substance of improvement. It yields the 
knowledge about the learning process and the social 
setting for learning and education, and encourages 
the refinement of objectives and the development of 
materials and instructional processes which incorpo- 
rate that knowledge into improved educational prac- 
tices, 


The implementation of those practices in the 
school systems across the nation - the differential 
accomodation of different institutions and agencies 
to {һе new roles and expectations created by the 
improvements – however, is a very different kind 
of problem and requires Strategies and tactics com- 
mensurate to the task as it is revealed. The differ- 
ences between the two processes and the unique 
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Characteristics of each may very well shape signifi- 
cantly the strategies adopted for one or the other. 
The functional demands of educational research and 
development, for example, place certain constraints 
on the process of implementation. Similarly, the 
exigencies of the process of implementation may 
impinge upon the strategies for R and D which may 
be adopted. 


RESEARCH, DEVELOPMENT, AND THE PROCESS 
OF IMPLEMENTATION 


Recent policy-level discussions of educational 
research and development within agencies respon- 
sible for research funds, between such agencies 
and their constituencies, and within the research com- 
munity itself, have made it clear that there is far 
from unanimous agreement on the definition of the 
terms commonly applied to research and develop- 
ment, For this reason, we believe it would be 
helpful to establish how we are now using various 
terms - not to develop for all time a thesaurus as 
applied to the research process, but rather to clar- 
ify our concepts and interpretations. 


The research and development process involves 
all those activities designed to produce knowledge 
about the learning process (research, for example, 
designed to improve learning theory or to improve 
our understanding of the fundamental chemical, bio- 
logical, and neurological processes underlying 
learning). It includes other kinds of activities de- 
voted to the social context of learning (for example, 
the classroom or school as a social system, or the 
relationship among non- instructional factors such 
as motivation of peers, socio-economic status, edu- 
cational level of the home and the direct instructional 
interventions of the school such as curriculum, in- 
Structional methodology, facilities and so on). It 
includes studies pertaining to the organizatiori and 
administration of the instructional enterprise. 


Research and development also includes the de- 
velopment of new practices, materials, and proc- 
esses based on the knowledge newly created in the 
other kinds of studies. This involves identification 
of the specific improvement desired and then the 
systematic engineering of the improvement, using 
allthe information derived frombasic research that 
is relevant. 


A third part of the process, closely related to 
development, is the demonstration of the feasibility 
of the new materials, processes, or organizational 
forms in other than the hothouse setting of the de- 
velopment laboratory. 


Finally, a fourth element in the R and D process is 
the systematic dissemination of information relating 
toallthedifferentstages. Information about research 
Projects completed and underway is essential to the 
basic researcher. The educational developer also 
needs to know the latest findings about learning if he 
is to be able to incorporate them in his work, Re- 
Searchers and educators in the schools need to know 
what the development people are upto. On the 
other hand, the kinds of problems encountered by 
those engaged in curriculum or hardware develop- 
ment may suggest research projects to the psychol- 
ogists and sociologists. School people need to keep 


aware of new techniques and possibilities under de- 
velopment. Demonstrations need to be publicized, 
too, and the success or lack thereof provides 
further information to both researchers and develop- 
ers. Finally, the existence of the dissemination 
capability itself needs to be widely publicized to in- 
sure that all those having the need can make appro- 
priate use of it. 


Applied to any given piece of knowledge or any 
given improvement in one or a few schools, the 
R and D process and the implementation of improve- 
ment process appear to be identical. The differences 
between the two processes begin to become visible, 
however, when the systematic implementation of 
improvements in the entire educational system isthe 
desired objective. The systematic and continuing 
introduction of improvements in education through 
judicious innovation and change based on research 
requires careful attention to the different roles played 
by the many independent agencies and institutions in 
education. 


The first attempt, for example, to implement a 

new curriculum in economics in a secondary school 
undertaken with the aid of the original project staff 
responsible for developing it can easily be viewed 
as the normal demonstration responsibility of the 
project. But if the scope of responsibility is enlarged 
to include developing the capability of any teaching 
staff anywhere in the country where a decision to 
implement that curriculum has been made (that de- 
cision assumes, of course, knowledge about the cur- 
riculum and a judgement as to its suitability), the 
problem is clearly larger than an individual project. 
Attention needs to be paid to the problems of devel- 
oping mechanisms for the inservice and preservice 
training of teachers touse the new curriculum. That 
effort in turn may point to the particular responsi- 
bilities of teacher training institutions across the 
nation both to assist in the particular effort and to 
insure, more generally, that preservice training is 
undertaken in the light of the almost certain need 
for continuing education of teaching staff to keep up 
with new departures in curricular application. Thus 
we have moved in this one instance very quickly away 
from the individual school setting to the much larger 
question of how teacher training institutions — and 
maybe, therefore, also state teacher certification 
agencies — are to play their particular and general 
roles of enhancing the likelihood of the adoption of 
proven innovation. 


The matter can be carried still further, School 
boards should be apprised of the need to continually 
re-examine curricular objectives in terms of pre- 
sent and future social needs;asa conséquence they 
can be expected to play a significant part in the net- 
works of differing responsible agencies having uniqu 
roles in the process of implementing educational а 
provements based on and validated by research Th 
responsibilities of industry in helping to devel à 
materials, in manufacturing, and in selling th "d 
similarly need careful attention. T heir res s ce 
ene vg for training their customers ae ihe 
use of new eveloped i 
LAN ped materials also need to be 


Viewed on a larger Scale, then, 
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implementation of solid innovations System-wide de- 
pendon factors more closely related tothe dynamics 
of social change than to the actual research and 
development process itselí. This dynamic- 
whether called administration, the politics of change, 
or continuing attention tothe intricacies of the proc- 
ess of building conviction inand a demand for more 
effective materials and practices in the schools – 
isa necessary condition for the implementation of 
the products of R and D in Schools. It constitutes, 
therefore, an absolutely essential problem for edu- 
cational R and D Strategists. 


RESEARCH DEVELOPMENT AND DISSEMINATION 
STRATEGIES OF THE PAST 


In describing the Strategies of the past, present, 
and future we have elected to characterize broadly 
in generalized form the major outlines of attack as 
pursued until a very few months ago, the kinds of 
changes which took place and that have placed us in 
а crucial period of transition, and the sorts of di- 
rections we project for the future. 


The availability of millions of dollars for educa- 
tional research and development is a relatively new 
phenomenon. The recency of this growth and its 
absolute size give some pause for thought. A re- 
view of developments will help disclose the range 
of options available to us now and in the future. 


The first piece of legislation Specifically author- 
izing the United States Office of Education to support 
research, surveys, and demonstrations in educa- 
tion — The Cooperative Research Act - was passed 
by the Eighty-third Congress іп 1954, In effect, the 
Congress said that research in education was impor- 
tant enough that the nation needed legislation to 
authorize Federal Support for it. The Act itself, of 
Course, represented a first Step in the development 
of strategies for conducting R and D to improve 
education, Four years later, in 1958, the Congress 
included two titles inthe National Defense Education 
Act (NDEA) authorizing research on the uses of new 


^. 4,7 а Special foreign currency program based 
9n the distribution of surplus foods. m 


аце passage and funding of Federal legislation 
parture Ta research in education was a major de- 
Convictio Rndof Itself. Ewas recognition of the 
toa irt that Federal support needed to be directed 
of individual bearing dimensions larger than those 
h financial at es and calling upon resources, 
means of st, and human, then currently beyond the 
ate and local educational agencies. 
The 


5 in Support of educational re- 
e identification of areas of con- 
technically excellent propos- 


Бе to a general call. In the 
Се, for exam 


Support of 


basis. Areas of importance were identified periodi- 
cally and interest maintained in those areas, but 
the guiding principle for the support of research was 
the technical excellence of proposals rather than the 
type of research or the substantive area of concern. 
That principle was viable because in the early days 
of the research effort it was always possible to fund 
all the technically excellent proposals without regard 
to the field of study. 


Over the years, substantial amounts of applied 
research as well as basic research were stimulated 
and supported. In more recent years the research 
program has moved into two kinds of activity: cur- 
riculum improvement and the establishment of re- 
search centers designed to explore programmatically 


Капар ina defined problem area in education. 


Even with the addition of these new thrusts the same 
principle mentioned before was still applicable. The 
field was young and growing, but it was still possible 
to support all of the quality proposals submitted. 


THE RECENT PAST AND 
THE PRESENT 


In 1965, several groups including staff inside the 
Office of Education, a Presidential Task Force, and 
others began to take a hard look atfederally supported 
research in education, then nearly 9 years old. The 
wisdom of earlier Strategies was apparent. They 
had been successful — in part because of the avail- 
ability of dollars, in part because of the generally 
unsolicited mode of operation - in stimulating the 
growth of and interest in research and development. 
The field was growing; the submissions of proposals 
was increasing rapidly (in fact, at a faster rate than 
appropriations); the educ ational community asa 


whole was beginning to pay attention to the possibili- 
ties of research. 


IMPORTANT RESEARCH DEVELOPMENTS 


Research and Development Centers. The newly 
acquired visibility of research and development, 
however, led to concern over some of the apparent 
deficiencies of the total research effort as it had 
emerged to that point. First, it became apparent 
that the individual research projects as a whole — 
while the acceptable levels of quality individually — 
did not fit together well enough to be considered 
coordinated approaches to Substantive problems in 
education. The Research and Development Centers 
were in part created in response to this need, but 
it was felt that major chunks of the research effort 
Should be better coordinated and designed to lead to 
cumulative results. 


Second, part of the difficulty in developing highly 
coordinated cumulative research efforts could be 
attributed to the inadequacy of the dissemination of 
information to the educational research community 


Third, a 
research in terms of Servic 
e 
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implementation. First, the stage of development was 
not being supported to anything near the degree that 
it should. Second, very little attention was being 
directed to the change process whereby improve- 
ments could be actually implemented in wide- 
spread fashion. 


Fourth, it was clear that the human resources 
available for R and D activities would need to be 
expanded in at least two ways. Training programs 
to develop new talent would be required. New 
agencies and institutions previously not eligible for 
support would need to be tapped. 


Amendments to Research Acts. The response to 
these felt needs took the form of a series of amend- 
ments to research legislation, the creation of a 
series of new program efforts under the aegis of the 
newly acquired authority, and the passage of a new 
piece of legislation. The basic piece of research 
legislation administered by the Office of Education, 
The Cooperative Research Act, was amended in 1965 
by Title IV of the Elementary and Secondary Educa- 
tion Act (ESEA). The amendments broadened the 
existing authority – to support research, surveys, 
and demonstrations in education — to include dis- 
semination. The range of eligible institutions was 
expanded to virtually all kinds of public and private 
organizations whether profit or non-profit. Author- 
ity was included to develop programs designed to 
train educational research and related personnel, 
and to upgrade training programs. The Office was 
authorized to award grants as well as contracts, 
and the Commissioner was given authority to award 
funds for the construction and equipping of facilities 
for research and related purposes. 


These amendments vastly extended the range of 
activities possible under the research program, and 
made it feasible for the research program to meet 
directly some of the needs identified in the reviews 
of the program to that time. In addition, the testi- 
mony before the Congressional committees - or the 
legislative history, as it is called — made it very 
clear that the broadened authority was to be used 
to bridge the gap between researchand practice and 
to pay substantially more attention to the problems 
of implementing the knowledge derived from the 
research efforts to date and in the future. The 
broadened responsibilities represented by these ad- 
ditionalauthorizations have created new demands on 
the administration of research and at the sametime 
have offered new tools for meeting emergent short- 
comings of the research effort. 


Training Program. To date, three major new 


program thrusts have undergone spirited develop- 
ment. A training program designed to expand the 
corps of educational researchers has been developed. 
This program provides both program development 
grants and an array of training mechanisms ranging 
from institutes of short and long duration through 
Undergraduate and graduate training programs to 
а small postdoctoral effort (Clark, 1967). 


Educational Research Information Center. The 
Educational Research Information Center (ERIC) 
has been established with Central ERIC, twelve 
clearinghouses, and more to come. ERIC is de- 
Signed to provide to researchers and practitioners 


alike an information storage and retrieval mechan- 
ism to make available instantly in easily accessible 
indexed form all the research andrelated data 
relevant to a particular problem. Research in Edu- 
cation, the monthly publication of the ERIC system, 
has now published several issues and already the 
usefulness of this effort has begun to prove itself. 


Educational Laboratories, The third and largest 
development in the research program has awakened 
the interest, excitement, and the enthusiasm of the 
entire educational community from local schools, 
state agencies, and teacher training institutions to 
industry, scholars from the arts and sciences, and 
the lay public. We refer to the National Program 
of Educational Laboratories. Drawing from re- 
sources in regions extending across the nation, new 
institutions called educational laboratories have been 
created to bridge the gap between research and prac- 
tice. These institutions were created by represen- 
tatives of all the many agencies and institutions 
which play different roles in the implementation 
process. They are reflections of the conviction that 
it is not enough to do research; that research must 
be followed up by development project which, having 
established the desired objectives — whether curri- 
cular, instructional, organizational, professional, 
or technical — then move to the development of solu- 
tions drawing upon the best that research hasto 
offer. The laboratories have also been charged with 
the responsibility for active dissemination campaigns 
based on the successful development projects they 
and others engage in. 


We have encouraged the laboratories to conceive 
their responsibilities broadly — encompassing much 
more than merely the distribution of information, 
although that is a critically important function among 
many. Clearly, one of the important Steps in the 
diffusion of research-based improvement throughout 
the educational system is the establishment of demon- 
Strations. First time demonstrations of feasibility 
will be the direct responsibility of the laboratories; 
the more widespread diffusion of the successful in- 
novations will depend on the degree to which infor- 
mation about the innovation is distributed to various 
parts of the systems, the degree to which the innova- 
tion recommends itself to professionals, and the de- 
gree to which credible demonstrations of the new 
practice or curriculum are mounted in schools, This 
last part of the diffusion process - the establishment 
of real-life (not hothouse) demonstrations of the in- 
novation without the intervention of the inventor - 
represents a place where the labs can work with the 
state education agencies and local school districts 
to make use of Title III of ESEA, Such demonstra- 
tion efforts must be coordinated and well-conceived 
Laboratories can help only in their areas of com e- 
tence and program thrust and only by берен 
closely with state education agencies and the 1, 1 
School districts. Seal 
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and the same time e ngage in 1) major efforts to » 
develop new materials, practices and organizations 
using the outcomes of research; and 2) by utilizing 
the expertise brought to them by the involvement of 
different agencies and institutions, pursue courses 
of action which help to hasten the process of im- 
provement once tested innovations are available. 


The strategy for these three new program thrusts 
is straightforward. If there is to be an expansion in 
research and development, people will need to be 
trained or recruited from new areas to fill the de- 
mand. In order to build structures of knowledge and 
cumulative improvement, dissemination networks 
and the material which moves through them will need 
to be better systematized and improved. Finally, to 
fill the gap between research and practice, anew 
autonomous institution drawing institutionally and 
representationally on many resources in the educa- 
tion system has been created, 


TITLE Ш OF ESEA 


In addition to the new authorizations for research 
and the programs mounted as a result of them, the 
Eighty-ninth Congress passed Title III of the ESEA. 
This program, authorizing support for projects sub- 
mitted by local educational agencies designed to sup- 
plement existing school programs or toserve as 
models for existing school programs, was an inno- 
vation in itself. When (former) Commissioner Kep- 
pel testified before the House Subcommittee on the 
Elementary and Secondary bill, he told the Congress- 
men that he viewed Title III and Title IV together. 
Title IV was to be the means whereby the substance 
of educational improvement would receive increased 
impetus and attention; Title III would be the means 
by which local schools could inaugurate the kind of 
credible real-life demonstrations which would be 
convincing to their counterparts and become there- 
by one of the moving forces for the widespread adop- 
tion of tested innovation. Those responsible for 
Title III estimate that this past fiscal year fully 60 
percent of the money went to Support projects relat- 
ing to the functions of development and demonstra- 
tion. This year the program expects to do at least 
às well. Most important, for thefirst time local 
educational agencies were provided with the funds 
and the encouragement to experiment with new ideas. 


CURRENT DEMANDS AND 
STRATEGIES FOR THE FUTURE 


A thorough considera 
of educational research on 


Those five varia- 
ollowing para- 


VARIABLES RELATED TO STRATEGIES 


The Research Function 
has been Charged broadly wit 
€s relating to i 


е nature of le 
for it, 2) developing mater 


running from demonstration to dissemination to train- 
ing. in order to ensure that improvements find their 
way to implementation in operational settings. 


The Financial Demand. The financial demand, 
represented by the broad responsibilities indicated 
above, is difficult to estimate with precision, but 
its dimensions can be sketched. Recent experience 
accumulated by the National Science Foundation, the 
Office of Education, and the Office of Economic Op- 
portunity has shown that full-scale multi-media cur- 
riculum development can range in cost from $2 mil- 
lion to $6 million per course; even this expense is 
engendered by building curriculums for schools as 
they are presently structured. The costs for simi- 
lar development for newly organized institutions 
might well be more. In any case, however, it does 
not take very complicated arithmetic to multiply the 
above figure by the number of possible course ex- 
periences that might be developed for preschool, el- 
ementary, secondary, community col lege, under- 
graduate, graduate, and adult education. If one as- 
sumes, as we think is reasonable, that such curric- 
ula probably ought to be redeveloped every 5 or 7 
years it is not unreasonable to project an annual ex- 
penditure of a half billion dollars just for curriculum, 
hardware, and organizational development alone, 
The research effort to support that kind of develop- 
ment effort will add another $150 million to $ 200 
million to that figure annually. The cost of demon- 
Strations must also be added — perhaps as much as 
$300 million or $400 million annually. A rough rule 
of thumb for dissemination activities indicates that 
approximately 5 to 10 percent of the research effort 
ought to be spent in that area. We can add conserv- 
atively for that purpose another $50 million, The 
total is slightly more than a billion dollars annually, 
an amount which — if added to the current estimated 
expenditures of educational institutions for this fis- 
cal year - would constitute just 2.2 percent of the 
total. That rate of expenditure for research and de- 
velopment approximates the minimum percentage de- 
voted to R and D today by American ind ustry. It 
Strikes us as reasonable that the education industry, 
with estimated expenditures of $48. 8 billion in fiscal 
1967, should devote a similar proportion of effort 


to research and development in areas relevant to its 
missions. 


The Dollars Available. The actual dollars avail- 
able for the broad range of activities associated with 
the research effort have grown dramatically. They 
àre by no means equivalent to the demands briefly 
Sketched above, But major funding for research 
and related activities is still in its infancy. Signifi- 
cant educational research funding began only in 1957 
at a combined USOE/NSF level of $1.7 million and 
has grown this year to an annual level of $200to $220 
million including USOE research appropriations, the 
NSF Course Content Improvement Activities, other 
miscellaneous agency expenditures, and Title III of 
ESEA (assuming that the 1966 proportion - 60 per- 
cent – of the Title III, ESEA, activities going to de- 
velopment and demonstration holds for 1967), 3 


This level of funding, matched with 
of the conceivable demand 
and related activities, undi 
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Manpower Resources. The problem of manpower 


resources is of crucial importance and is not easily 
resolved. A direct response, although a long-range 
one, is to train a new corps of professionals having 
the kinds of skills required to perform the entire 
range of research functions. There are other pos- 
sible approaches, however. There already are- 
in other disciplines and in other kinds of institu- 
tions ~ individuals skilled in the research and devel- 
opment environment, and who could probably be per- 
suaded to turn their attention to the field of educa- 
tion. These considerations were in no small meas- 
ure part of the strategy behind broadening the listof 
eligible organizations to participate in R and D. They 
also played a major role in the early decision inthe 
administration of research to secure the active in- 
volvement and participation of arts and sciences 
disciplines in educational research, a strategy that 
has continued until now and willbe maintained in the 


future, 


The combination of approaches adopted for the 
provision of adequate manpower supplies can be ex- 
pected, then, to have considerable bearing on the 
possibilities for growth (see Clark, 1965; Guba and 
Elam, 1965; Hopkins, 1967). 


Successses of Past and Current Efforts. The 
successes of educational research and development, 
the growing visibility of operations in this area, and 
the increasing conviction that research will in the 
long run provide the basis for substantive improve- 
ments in education have created new demands upon 
the human and financial resources available to the 
research enterprise. As soon as the demand out- 
stripped the resources, something that happened 
only very recently, the entire picture of support 
necessarily was altered. It has now become neces- 
sary to allocate limited resources in the face of de- 
mands which exceed those resources. 


ADDITIONAL STRATEGIES ADOPTED 


The five considerations discussed above have led 
to the adoption of additional strategies. Broadly 
speaking those strategies embrace the three princi- 
pal functions discussed below. 


Careful Long- Range Planning. Now that the de- 


mand has clearly out-stripped the supply of re- 
sources, particularly in the financial area, it 15 
critically important that the allocation of research 
dollars to various research functions and to partic- 
ular topics or areas of development be accomplished 
in a careful and logical manner. This must be done 
in ways to ensure that both short-term and long- 
term interests are met, and to maintain and 
Strengthen the existing political structure of educa- 
tion as it has developed over the years. 


The careful long-range planning of those re- 
Sources is much more likely to produce optimum 
levels of support for all the different functions and 
areas for research and development than is a totally 
non-directive approach embracing the competitive 
selection of technically excellent but unsolicited pro- 
Posals for support, Such planning will need to take 
into consideration the instructional and administra- 
tive needs of the different educational levels and the 
many kinds of agencies, institutions and organiza- 


tions in the field of education, industry, the profes- 
Sions, and government. Attention will have to be 
given to the needs of different target populations, in- 
cluding the general student, the handicapped, the dis- 
advantaged, and the gifted. This implies, of course, 
that it will be necessary for us to develop much 
more effective mechanisms for planning research 
and related activities in order to ensure that all rel- 
evant data are canvassed and that maximum flexibil- 
ity is built into the planning and administrative pol- 
icies devolving from that planning. We must be 
careful, for example, that we continue to pursue 
the wisdom of earlier efforts in support of research 
by maintaining a significant portion of the research 
budget for unsolicited efforts. This is only one ele- 
ment of the larger planning picture, howe ver, al- 
though a very important one. 


As part of the attempt to improve the data on the 
basis of which research and other educational plan- 
ning is now done, the Bureau of Research has re- 
cently called for proposals to begin pilot projects 
we hope will ultimately lead to the establishment of 
educational policy centers. These centers would, 
on a continuing basis, conduct the kinds of studies 
which would enable all of us to have better ideas, of 
the 5-, 10-, 20-, and 30-year picture for education. 
How should or might the schools relate to the soci- 
ety? What should the range of curricular objectives 
be? What kinds of sources might exist for financing 
education? How might schooling be structured in the 
Society in the years to come? What technologies 
are likely to be available? These centers will not 
be engaged in predicting what the future will be. In- 
stead, they will try to project the many alternatives 
available to us, cost them out, examine the conse- 
quences, and thus provide much better data to policy 
planners than are now available. They will provide 
а new kind of information and analysis looking to the 
future in a systems-oriented interdisciplinary way. 
Supplementing the improved present-oriented data 
collection which must also accompany the planning 
process (see Goodlad, 1967). 


The Outputs of Research Process. Our concern 
for the outputs of the research process is pragmatic 
in part, but it is tied closely to our conception of the 
mission of the research effort in education in gen- 
eral and of USOE's research program in particular. 
We have stated our faith that the improvement of ed- 
ucation depends in the long run on th е systematic 
prosecution of a well-conceived R and D effort. The 
improvement of education is the fundamental thrust 
of the mission; that criterion must be the continuing 
evaluative guide we employ in judging the success 
of our efforts. 


public money can only be the eventual and signifi 


betterment of the educational system as а whole, it 
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There can be no denying the growth and the 
strength of the interest in results. The Congress 
is concerned. The Executive Branch is concerned. 
Local school people have made their interest felt in 
a number of ways, particularly in the development 
of the programs for the educational laboratories. 


We believe, however, that a strong and continu- 
ing tension exists between the conduct of research 
designed to increase our knowledge of basic learn- 
ing procésses, for example, and the pursuit of de- 
velopment projects designed to yield specific out- 
comes for instructional use. The demand for re- 
sults has raised the appropriate and throughly justi- 
fiable concern of the academic and scholarly re- 
search community that such demands may tend to 
compromise the long-range efforts which are so 
badly needed. The continuing tensionbetween short- 
term and long-term requirements - between today's 
youngsters and tomorrow's — isrealand constitutes 
one of the continuing nightmares of the research ad- 
ministrator. This problem is the responsibility of 
the administrator of research, and it must be square- 
ly met. We believe that it is important that the two 
functions not be confused, but that they both be 
served with sophistication and energy. 


The importance of concentrating on the improve- 
ment of schools, and the significance of development 
projects in that effort, signals an increasing atten- 
tion to such activities as curriculum development 
and explicit attempts to improve instructional prac- 
tices. We are increasingly persuaded, for example, 
that prudent management of the limited resources 
available at the present time makes it necessary for 
us to adopt a research strategy that relates applied 
research projects closely to identified development 
efforts. We would hasten to add that the preserva- 
tion of a signficant portion of funds for unsolicited 
proposals will still permit the support of some ap- 
plied research unrelated to development since good 
ideas will also be generated independently of develop- 
ment projects or research planning groups and 
should be supported on their own merits. 


We anticipate, then, still further increase in at- 
tention to development kinds of tasks desi gned to 
produce substantial, measurable, andcumulative im- 
provements in the nation’s schools. 


Continued Expansion of Research and Related 
Activities, There is no question inour minds, as 
all the preceeding analysis should clearly indicate, 
that research and related activities must continue 
to expand in rapid but judicious ways. By directing 
our attention to the development of coordinated strat- 
egies for the support of research with the implica- 
tion that greater attention will be paidto the out- 
comes of the research effort, we believe that con- 
Stituencies in Support of educational research will 
grow and the dollars available for the function will 
Increase to what we belieye are the regular mini- 
mal levels we Suggested earlier. 


SUMMARY 


In this paper we have Sketch 

ed out the two t: 

of the change process: research and development 

e "i one hand, and implementation, on the other. 
ave suggested that the Principal purpose of the 


U 


research effort ought to be viewed as the improvement 
of education in the schools. We have indicated the 
national need for research, and the importance 
of drawing on national resources to meet that need. 
We have described the discrepancy between the po- 
tential size of the need in dollar terms and the actual 
fiscal resources available. We have suggested that 
these conditions lead us to adopt, in effect, multiple 
strategies for R and D. These include: 1) better 
planning of the research effort; 2) more attention to 
the training of research and related personnel; 3) the 
establishment of new institutions — the laboratories — 
and new programs – Title III - to greatly increase 
the opportunities and responsibilities of local school 
officials in the research and implementation proc- 
ess; 4) the continuation of significant proportions of 
unsolicited research and development funds; and 5) 
increased attention to development efforts designed 
to yield better materials, equipment, and instruc- 
tional practices for the schools. All of these strat- 
egies will, we hope, lead to increased understanding 
of the ultimate impact of the research effort, help 
to bring about substantial increases inthe amount of 
Support committed to research and development in 
education, and will provide the foundation for the con- 
tinuing improvement and self-renewal of our educa- 
tional system. 


NOTE: Dr. R. Louis Bright is no longer with the 
United States Office of Education. He is presently 
located at Baylor University in Waco, Texas where 
he is a professor. 


FOOTNOTES 


1. Digest of Educational Statistics, 1966, U. S. Gov- 
ernment Printing Office, Washington, D.C., 
1966, p. 17. 


2. The imprecision in the estimate derives from a 
number of considerations. The research au- 
thorizations of USOE in FY 1967 total $99 mil- 
lion, but $6 million are for training and an- 
other $14 million are for construction. The 
National Science Foundation spends close to 
$17 million a yearoncurriculum development, 
and almost certainly some of the curriculum 
development efforts of the Office of Economic 
Opportunity will be used by schools. Similar- 
ly, the National Institute of Mental Health and 
National Institute of Child Health and Human 
Development also spend a considerable amount 
on research related to areas of interest to 
learning and education. Title III of ESEA, if 
past percentages hold, could well spend $81 
million on development and demonstration ac- 
tivities. Part of the development of Strategies 
for educational R and D is the coordination of 
these several sources of research and re- 
search related funds. 


3. The actual figures by program and year are as 
follows: (See chart on following page. ) 


E 


1954 
1955 
1956 
1957 
1958 
1959 
1960 
1961 
1962 
1963 
1964 
1965 
1966 
1967 
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USOE NSF Course Title ПІ, ESEA 
Research Content Dev. and Dem. 
$ 1,725 E 
35, 000 

ВРЕ 18,000 
1,020, 000 650, 140 
2, 300, 000 750, 310 
6, 716, 000 6, 180, 485 
10, 350, 000 6, 302, 055 
10, 117, 750 6, 167, 740 
11, 770, 000 9, 389, 948 
14, 188, 400 12, 626, 771 
19, 820, 000 14, 157, 650 
37, 703, 000 14, 889, 081 ады 
100, 141, 241 16, 393, 383 45, 400, 000 
99, 600, 000 17,000, 000 (est.) 81, 000, 000 (est. ) 
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^ The Wisconsin Research and 


Development Center 


for Cognitive Learning 


THE AGREEMENT between The Uni- 
versity of Wisconsin and the United States Office of 
Education (USOE) establishing the Wisconsin Re- 
search and Development Center for Cognitive Learn- 
ing was signed on August 6, 1964. The high expec- 
tations held for the Center were expressed by the 
principals to the signing in a number of ways. 


University President Fred Harvey Harrington 
remarked that the signing denoted **... an impor- 
tant day in the history of the University. This agree- 
ment marks the culmination of years of effort de- 
voted to the improvement of the research capabili- 
ties of the School of Education, and opens the way 
to still greater developments. '' 


Howard F. Hjelm, then director of basic re- 
Search for USOE, described the agreement as fol- 
lowing the new pattern in educational research of 
full institutional commitment. He stated: “Тһе Uni- 
versity has pledged itself to fully investigate the 
problem of learning and to disseminate research 
findings in a way which will bring about definite 
changes in school practices, ’’ 


Hjelm found the University well suited for the 
commitment it had accepted. ‘‘The University was 
chosen because of the exceptional quality of its edu- 
cational researchers, because of its excellent rela- 
tionship with local schools and with the State Depart- 
ment of Public Instruction, and because of the whole- 
hearted dedication to the project existing in Madison. 


Center Director Herbert J. Klausmeier stated 


the primary focus of the Center: **Research empha- 
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mem Herbert J. Klausmeier 
The University of Wisconsin 
Madison, Wisconsin 


ses will be on learning by children and youth in nor- 
mal school situations. Especially stressed will be 
the learning of concepts and problem-solving tech- 
niques in mathematics and other basic subjects. '' 


The Wisconsin R and D Center was established 
under provisions of the Cooperative Research Pro- 
gram of USOE. In 1964 the Cooperative Research 
Program included six major areas: 1) basic and 
applied research, 2) demonstration, 3) curriculum 
improvement, 4) small contract, 5) research and 
development centers, and 6) developmental activi- 
ties. Further description of the research and de- 
velopment centers was given inOE Publication 
12017, 1963. as follows: 


Research and development centers are de- 
signed to concentrate human and financial re- 
sources on a particular problem area in edu- 
cation over an extended period of time in an 
attempt to make a significant contribution to- 
ward an understanding of, and an improvement 
of educational practice in, the problem area, 
More specifically, the personnel of a center 
will: 

1. Conduct basic and applied research studies, 
both of the laboratory and field type. 

2. Conduct development activities designed to 
translate systematically research findings 
into educational materials or procedures, 
and field test the developed products. 

3. Demonstrate and disseminate information 
about the new programs or procedures 
which emerge from the research and devel- 
opment efforts. These activities may 
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include demonstrations in a natural, or 
operational, setting; the preparation of 
films, tapes, displays, publications, and 
lectures; and the participation in symposia 
and conferences. 

4. Provide nationwide leadership in the chosen 
problem area. 


The preceding ideas about the programmatic re- 
search, development, and dissemination thrust of 
an R and D Center in relation to a significant prob- 
lem area in education followed logically from expe- 
rience with other programs of the Cooperative Re- 
search Program of the Office of Education, thefirst 
of which, basic and applied research, started ona 
modest scale in 1956. Also, the research and de - 
velopment strategy had demonstrated a high yield 
in selected aspects of industry, agriculture, medi- 
cine, and space exploration. A large segmentof the 
American public desired improved educational prac- 
tices. Relevant USOE, university, and other con- 
cerned persons throughout the nation felt thata pro- 
gram of research and development centers would 
be equally effective in improving education. 


An R and D Center, designed to improve educa- 
tional practices through programmatic research 
and development, might be considered as a concept. 
As we know, concepts are abstractions of realities. 
A concept is comprised of the attributes, or proper- 
ties, of things and events, the attributes being 
joined by rules, or ways of combining attributes 
Systematically. An R and D Center then may be 
understood in relation to various attributes and how 
they are combined to form a whole. The writer hy- 
pothesizes eight crucial attributes of an R and D 
Center as follows: problem area, outcomes sought, 
research and development program and strategy, 
dissemination and adoption program and strategy, 
staff, administration and management, locus of op- 
eration, and source and stability of funding. The 
preceding attributes may be considered as the major 
components of a system, or instrumentality, de- 
signed to achieve certain general objectives implied 
by the problem area. Thus, a systems approach is 
properly followed in organizing and managing the 
Center operations and programs. The remainder of 
this chapter is devoted to a discussionof the various 
components of the system, their relationships, and 
the outcomes attained thus far by the Center staff. 


Before further description of the Wisconsin Н 
and D Center, it should be recalled that the Elemen- 
tary and Secondary Education Act of 1965 as noted 
by Bright and Gideonse in the previous chapter, pro- 
vided for the establishment of regional educational 
laboratories and Title III demonstration and ado p- 
tion centers in local schools. These two programs 
of USOE are much larger than the current R and D 
Center Program. As instrumentalities for improv- 
ing educational practices, the writer hypothesizes 
that they, too, can be described and understood in 
terms of the preceding eight attributes of a center. 
It is not the purpose of this chapter, however, to 
Point to the similarities and differences among the 
various research, development, and demonstration 
instrumentalities of the USOE but to describe the R 
and D Center of Wisconsin as an example of a signil- 
icant contributor to the extensionof knowledge about 
cognitive learning and the improvement of related 


educational practices through research and devel- 
opment. 


PROBLEM AREA 


The problem area of the Center is the improve- 
ment of education through a better understanding of 
cognitive learning. The importance of this problem 
area may be inferred from five propositions regard- 
ing learning and instruction. 


The first proposition is that concepts provide 
much of the basic material for thinking. Concepts 
possessed by the individual enable him to interpret 
the physical and social world and to make appropri- 
ate responses. Concepts once learned then serve 
as symbolic mediators between sensory input and 
overt behaviors. Without concepts with which to 
think, man, like lower-form animals, would be lim- 
ited mainly to dealing with sensorimotor and percep- 
tual representations of reality, closely tied to im- 
mediate sensory experiences. 


A second proposition is that concepts and cogni- 
tive skills comprise the major outcomes of learning 
in various subject fields. Sentence, noun group, 
and transforms represent four concepts in English 
syntax. Set, number, and fraction represent con- 
cepts in mathematics. In order to learn the con- 
cepts of a discipline, a student must acquire cogni- 
tive skills. In mathematics these skills are desig- 
nated by such terms as arithmetic comprehension 
and problem solving. The identificationof concepts 
and their arrangement in a hierarchical order and 
the identification of cognitive skills are principal 
concerns of scholars and curriculum workers in the 
various subject fields. 


A third proposition, related to the preceding 
ones, is that the acquisition of cognitive skills en- 
ables man not only to acquire the knowledge of oth- 
ers but also to generate new knowledge. Cognitive 
processes, such as cognizing, hypothesizing, and 
evaluating information, are learned. These and 
other processes, combined with cognitive contents, 
are designated cognitive abilities or skills, When 
the child has learned the cognitive skills of reading, 
he can acquire knowledge in the absence of other per- 
sons. When he has mastered mathematical concepts 
and computational skills, he can find solutions to 
problems not encountered inthe same form by any 
other person. 


A fourth proposition is that efficient learning of 
concepts and cognitive abilities is related to condi- 
tions within the learner and conditions within the 
situation. Two conditions within the learner asso- 
ciated with efficiency of learning are attention re- 
lated to motivation, and level of cognitive function- 
ing. In school settings, the preceding conditions 
related to the student and other conditions related to 
the content and sequence of instruction, quality and 
availability of equipment and materials, and quali 
and methods of instruction are paramount. Mor x 
ficient learning can be attained through better et 
trol of these conditions. TRU 


? A final proposition is that knowledge ab 
tive learning, concepts, cognitive skills 
ditions of learning generated by scholars 
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validation in school settings. Although the study of 
cognitive learning is in its infancy, sufficient knowl- 
edge has been generated to develop instructionalthe- 
ory regarding the teaching and learning of concepts. 
The validity of the theories must be tested in school 
settings. Similarly, instructional materials for 
children and outlines of concepts for teachers must 
also be tested for usefulness and validity in school 
settings. 


PROGRAMS, STRATEGIES, 
AND OUTCOMES 

Strategies best suited to achieve the Center goals 
include the differentiation of the problem area into 
research and development programs and other pro- 
grams. The three research and development pro- 
grams are titled Processes and Conditions of Learn- 
ing, Processes and Programs of Instruction, and 
Facilitative Environments; the other programs deal 
with Dissemination, Training Activities, and Tech- 
nical Support. These programmatic activities are 
designed to concentrate and utilize human and mon- 
etary resources most effectively over an extended 
period of time. Each research and development 
program is characterized by a sufficiently sharp 
focus to assure its identity and by a relatedness of 
content and methodology that facilitates the free flow 
of information and personnel among programs. Fig- 
ure 1 shows the major operation of the Center and 
the program components. These operations and 
components are now described in some detail, with 
attention first directed to the three Rand D pro- 
grams and later to the other programs. The objec- 
tives of each program are given and outcomes at- 
tained through 1968 are summarized. 


PROGRAM 1 -- CONDITIONS AND PROCESSES OF 
LEARNING 


To increase our understanding of conditions and 
processes of cognitive learning, basic research is 
needed. To test in school settings the knowledge 
from basic research, relevant models and systems 
must be developed. To meet these requirements, 
the research and development activities comprising 
Program 1 are directed toward: a) generating new 
knowledge about concept learning and cognitive skills 
by conducting laboratory-ty pe experiments or re- 
search within the school setting; b) synthesizing ex- 
isting knowledge and developing general taxonomies, 
models, or theories of cognitive learning; and c) de- 
veloping educational materials suggested by the pri- 
or activities. 


Substantive information from the basic studies in 
learning feeds directly into the area of instructional 
improvement, thus providing from the behavioral 
sciences a foundation for the development of instruc- 
tional programs. Also, methods developed in the 
behavioral sciences are utilized in the researchand 
development activities of Program 2. At the same 
time, the behavioral scientist in Program 1, when 
dealing with the learning of content in a subject field, 
utilizes personnel from Program 2 and other 
sources who are expert in the subject field, familiar 
with instructional processes, or both. 


Basic research in cognitive learning has been 
conducted by the Center staff and reported regard- 
ing concept léarning, creativity, prereading skills, 


rule learning, the shaping of complex human behav- 
ior, group pressures on individual performance,and 
the transmission properties of various media. Rel- 
evant results from these studies have gone into the 
development of instructional programs. Inaddition, 
a system of motivation was formulated and put into 
operation in selected elementary schools during 
1967-68. Creativity training materials and proce- 
dures were prepared and received initial testing in 
selected schools in 1967-68. Procedures for teach- 
ing severely retarded readers were developed and 
field tested, starting in 1965, and were used opera- 
tionally in selected schools in 1967-68. Other basic 
research dealing with concept learning and cognitive 
skills is continuing and is being applied to improve- 
ment of instructional programs in arithmetic, Eng- 
lish, reading, and science. 


PROGRAM 2 -- PROCESSES AND PROGRAMS OF 
INSTRUCTION 


What is an appropriate unit to deal with when at- 
tempting to improve instruction, or stated differ- 
ently, to achieve greater efficiency of student learn- 
ing? The Center staff and school people decided that 
it is each building or an integrated set of buildings 
which the same children attend rather than individual 
children, separate classrooms within a building, or 
all the buildings of a local sehool system. Within 
each building, the instructional and other staff must 
deal with all the components of an instructional sys- 
tem. Similarly, the R and D staff, particularly 
those in Program 2, are concerned withallthe com- 
ponents of an entire instructional system, although 
they may give much more time and effort to devel- 
oping and testing one component rather than to 
another. Figure 2 outlines the major components of 
a total instructional system that are relevant to 
a school building. The order from top to bottom 
also indicates the general sequence in which the 
staff of the R and D Center and schools move in de- 
veloping a prototypic system, such as in reading, 
mathematics, science, and English. Variables as- 
sociated with each component require research. 
Thus, research on instructional processes and the 
development of instructional programs are essential 
inthis program. The specific objectives of Pro- 
gram 2 are: a) to establish a rationale and strategy 
for developing instructional systems in the cognitive 
domain; b) to identify by careful synthesis and 
further research sequences of concepts and cogni- 
tive skills within and across disciplines; c) to devel- 
ор assessment procedures and materials for the con- 
cepts and skills identified above; d) to identify exist- 
ing materialsor developnew instructional materials 
associated with the concepts and cognitive skills;and 
€) to generate new knowledge through research about 
instructional procedures including motivation, indi- 
vidualization, classroom management, and organi- 
zation of instruction. 


Executing the activities of this program requires 
specialists in a subject field, in instructional meth- 
ods, or both, with assistance from behavioral sc i- 
entists and communication experts. Each project 
in this program has at least one staff member whose 
speciality is cognitive learning. In addition, the 
program utilizes the many resources outside the 
Center including the University educational TV sta- 
tion (WHA-TV) and Educational Testing Service. 
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Figure 1: Major operations and components of the Wisconsin R and D Center for Cognitive Learning 
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The largest instructional program developed by 
the Center staff through 1968 was Patterns in Arith- 
metic, a program comprised of TV video tapes, stu- 
dent lessons, and teacher notes. It is designed as 
a complete program of materials for children and 
also for simultaneous in-service education of teach- 
ers. As of 1967-68 the materials for Grades 1-4 
had been developed and initially tested and were un- 
dergoing further field testing in many schools of 
Alabama and Wisconsin. The program for Grade 5 
was being completed. The development of a proto- 
typic instructional program, not using TV proce- 
dures and materials, dealing with concepts in math- 
ematics, Kindergarten-Grade 6, was started in 
1967-68. 


Fundamental research was conducted and re- 
ported on the learning and teaching of reading, 1966- 
68, and is continuing. An outline of major content, 
behavioral objectives, and related appraisal proce- 
dures was developed, Kindergarten-Grade 6, and 
relevant instructional materials and procedures 
were identified. The entire prototypic instructional 
System was piloted in selected elementary schools, 
1967-68, as cooperative school and Center activ i- 
ties, with substantial support of ESEA Title III and 
Title I funds, 


An outline of concepts related to the particle na- 
ture of matter and environmental conservation, ap- 
propriate for the elementary-school science pro- 
gram, was developed, 1965-68. Research on in- 
structional methods was reported and other research 


is continuing. Preliminary instructional materials 
and methods appropriate to the concepts were devel- 
oped during 1967-68 and initially tested. Related re- 
search and the development of scientific equipment 

and materials is continuing. 


A prototypic program of instruction in English in 
the junior high-school was under development, start- 
ing in 1966. An outline of selected concepts of Eng- 
lish syntax and a related set of programmed instruc- 
tional materials to teach the basic concepts were de- 
veloped and field tested during 1966-67, F urther 
testing of the programmed material, and implemen- 
tation of the entire curriculum, was started in selec- 
ted junior high schools during the 1967-68 school 
year and is continuing. Basic research on proces- 
ses and conditions of learning is continuing. 


An outline of concepts in verbal argument - 
veloped, 1965-67, and related teateoferitical thee. 
ing were also developed and initially validated, Pro- 
grammed material was prepared during 1967-68 to 
teach these concepts. Further research on the pro- 
gram and validation of the tests Continued, 1967-68 


Two other sets of activities have high prior it 
for initiation in subsequent years. Research ang ies 
lated development regarding social Studies in the 
elementary school are projected, Greatly increased 
use of the computer in research and in the man: 2 
ment of individually guided education is the se И 
field to be given increased emphasis, poog 
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Figure 2: Major components of an instructional system 


Students with entering behaviors 


Measurement Tools and Evaluation 


and characteristics 


v 


Instructional System Components 


An outline of content--cognitive, psychomotor, aífective-- 
to be learned. 


A statement of related behavioral objectives, or desired 
terminal behaviors of students, related to the content. 


Instructional materials, media, and consumable supplies 
to be used. 


Student activities-- one-to-one, small group, class-size 
group, and independen' study to achieve the objectives. 


Teacher activities--organization and direction of student 
learning activities, counseling and guiding students, 
classroom management, other functions. 


Procedures for placement and management of students. 
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Procedures 


To assess student's 
prior achievements 
or readiness to en- 
gage in specific ele- 
ments of the program. 


To ascertain student's 
intellectual abilities 


and other characteristics. 


Organization for instruction- -nongraded units. 


and equipment. 


Other: In school--other educational personnel. 
Out-of-school--home, neighborhood. 


Procedures for scheduling flexible use of time, space, 


To measure and evaluate 
student's progress 
during short and long 
intervals. 


V 


Students with terminal behaviors 
and characteristics 


PROGRAM 3 -- FACILITATIVE ENVIRONMENTS 


Achieving improved educational practices re- 
quires facilitative environments in selected local 
school buildings, in total school systems, and among 
various agencies. Thus, the research and develop- 
ment activities within this program are designed a) 
to develop and test new organizations and relation- 
Ships among staff that facilitate research and devel- 
opment activities on cognitive learning in schools 
and b) to develop and test the effectiveness of the 
means whereby schools Select, introduce, and uti- 
lize the results of research and development. The 
Same schools that facilitate Student learning also 


Serve as demonstration centers for enti 
ntire scho 
Systems and the R and D Center. ü 


In seven school systems of Wisconsin en viron- 

r shed, 1965-68, designed to facil- 

E the research and development activities of the 
enter and the learning of Children througha system 


To evaluate the separate 


components and the total 
system. 


of individually guided education. Anew organization 
for instruction that replaces the self-contained class- 
room by instruction and research units and an in- 
structional decision- making committee is an integral 
component of the facilitative environment of the Mul- 
tiunit School. The organization is shown in Figure 
3. The characteristics of individually guided educa- 
tion and of the organization which facilitates its ex- 
ecution merit further attention. 


The Multiunit School organization includes botha 
formal organizational structure and a procedural 
style consisting of several essential components as 
Shown in Figure 3. The organizational hierarchy of 
the Multiunit School consists of groups at three dis- 
tinct levels of operation. 


At the classroom level are the Instructional and 
Research (Land R) units. EachIandR unit һава 
unit leader or professional teacher, two or more 
regular staff teachers, one or more aides or secre- 
taries, and in some cases an intern. The number 


THE WISCONSIN RESEARCH AND DEVELOPMENT CENTER FOR COGNITIVE LEARNING 151 


Figure 3: Organizational chart of a multiunit school of 600 students 
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of instructional staff varies with the size of the unit. 
The intern assumes instructional responsibilities 
and does not perform routine and clerical duties. 
Each unit is charged with planning and conducting 
the total school experience of about 150 students in 
the prototype shown in Figure 3. 


At a second level of organization, the principal 
and the unit leaders constitute the permanent Instruc- 
tional Improvement Committee of the building. The 
principal chairs the group, which meets weekly, 
more often if necessary. This committee may bring 
in consultants from the central office, the state ed- 
ucation agency, or other agencies. Instructional de- 
cisions made by the committee are executed in the 
units. 


At the third organizational level is the Sy st e m- 
wide Policy Committee. Chaired by the superintend- 
ent or his designee, this committee includes re pre- 
sentative principals, unit leaders, teachers, consult- 
ants, and other relevant central office staff. The 
Personnel of this Committee have the capabilities 
and specialization necessary to assist the staff of the 
Multiunit School. The Committee meets less fre- 
quently than either of the other groups, but its oper- 


ation is one key to the success of the Multiunit 
School. 


The organizational pattern of the Multiunit School 
thus differs from that of the traditional, self- con- 


tained classroom school in several ways. First, in 

the Multiunit School personnel work in unitsor com- 
mittees, rather than in isolation as is the c ase in 

the traditional school. Second, three new roles are 
added: unit leader, teacher aide, and instructional 
secretary. Finally, the addition of new roles and 

the use of personnel in groups rather than alone re- 
sults in considerable redefinition of the familiar 

roles of principal, teacher, and consultant, 


In individually guided education in the Multiunit 
School, the Instructional Improvement Committee 
determines the objectives for the particular school 
building, taking into account system-wide and state 
regulations. These are broad institutional objec - 
tives for the school building. The Staff of each non- 
graded Instruction and Research Unit, the replace- 
ment for the graded homeroom or se lf - contained 
classroom, then decides the objectives for e ach 
child in the unit. While the unit leader takes the 
initiative here, each unit teacher also participates 
Assessment of the child's characteristics is through 
observation and by means of locally constructed and 
standardized instruments of various types. On the 
basis of the assessment each child is then Placed in 
one-to-one, small group, class-size group, and unit- 
size group activities. Instruction e mploy ing tu- 
torial work and computer-assisted instruction are 
examples of one-to-one activities, Activities in 
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small groups of two to fifteen are organized to at - 
tain socializing and also skill objectives. Inconnec- 
tion with skill objectives, 150 children in a unit 
might be placed in fifteen small groups for most of 
their mathematics instruction and then regrouped in 
another fifteen groups for part of their reading in- 
struction. Class-size or homeroom activities are 
used for achieving any objective where heterogene- 
ity is desired. Large unit groups of 40 to 150 are 
iormed mainly for giving information to the total 
group orfor independent study. The information is 
given by a teacher, television, Sound motion picture, 
or by other means. Some music and physical edu- 
cation activities are conducted in groups larger than 
the usualclass size. Independent study is carried 
out in small groups, class-size groups, and large 
groups. 


During 1967-68 approximately 4, 600 students in 
the elementary school, 1,600 in the junior high 
School, and 400 in the senior high school were en- 
rolled in R and I Units in Wisconsin schools. In 
these schools, children, educational practitioners, 
behavioral scientists, and other staff members of 
the Center are brought together in a close working 
relationship. Thus the selected schools of Wiscon- 
sin providing the facilitative environment comprise 
a primary link between improved school practices 
and Center research and development. 


DISSEMINATION 


The R and D Center is committed to disseminate 
the products of its research and development activ- 
ities to appropriate personnel by various media 
and also to install them in selected schools where 
others may directly observe the related educational 
practices. So that the Center may fulfill its obliga- 
tion of improving educational practice nationally, co- 
operation with personnel in key organizations of the 
state and nation is essential. An internal informa- 
tion section and an external network involving the 
Center and other agencies were established, start- 
ing in 1964, and are being extended in order a) to 
transmit information directly to scholars and others 
through Center supported publications, conferences, 
etc. ; b) to communicate information to personnel of 
selected school systems and school-related educa- 
tional agencies regarding the outcomes of the Cen- 
ter's programs as they become ready for dissemi- 
nation and school use; c) to distribute knowledge re- 
garding cognitive learning and products to agencies 
that function as linking mechanisms with schools; 
and d) to receive from the other agencies in the net- 
work information that may be used to improve the 
Center's products and operations. 


Reports of research and descriptions of develop- 
mental activities are disseminated through various 
publications. Tn addition, all instructional programs 
and procedures produced and tested by the Center 
may be observed in selected schools. The Center is 
working with other agencies to assure statewide and 


national dissemination and adoption of Center prod- 


uea, The State Departmentof Public Instruction of 
У сон, regional educational laboratories, Title 
вав Title Ш programs of ESEA, local schools of 

'isconsin and other states, agencies of The Univer- 
Sity of Wisconsin, and the p 


у rivate sector ar М 
volved in the dissemination network. US 
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The Center's publication program carried out by 
the Office of Information Services has four major 
components – Technical Reports, Theoretical Papers, 
Working Papers, a Newsletter, and books. Techni- 
cal reports describe the rationale, methods, and re- 
sults of original research conducted by Center staff. 
Research reviews, statements of theoretical po si- 
tion, and descriptions and rationale for project de- 
velopment are the content of theoretical papers. The 
theoretical papers are formal documents that, like 
technical reports, may be classified as monographs. 
Practical papers contain information of immediate 
significance to educational practitioners in establish- 
ing or extending instructional programs - guidelines 
for beginning new programs, outlines and specific 
procedures for instruction, evaluation summaries 
and procedures, and progress reports of projects 
conducted in the schools. Working papers present 
plans and procedures for initiating and executing re- 
search and development activities. 


Analyses of Concept Learning edited by Herbert 
J. Klausmeier and Chester W. Harris and the pres- 
ent book are collections of papers read and discussed 
atthe Center. The first, from a conference of the 
same title, was published by Academic Press in the 
fall of 1966. The second, from the Center colloqui- 
um series, was published by De mbar Educational 
Research Services in 1968. 


TRAINING ACTIVITIES 


Three conditions indicate the needfor a high 
priority to be given to the training of personnel in 
educational research and development in the R and 
D Center. First, the nation needs a large number 
of able persons in educational research and develop- 
ment. The many new programs under the ESEA of 
1965 andothers originating in the various states and 
local school systems require research and develop- 
ment personnel as well as knowledge and solutions 


to many educational problems. Second, The Univer- 
sity of Wisconsin has both atradition and the re- 
sources to contribute to the training of educational 

researchers, particularly beyond the baccalaureate. 


The University has strong graduate programs 
through which students acquire knowledge and skills 
essential for educational research. Third, theR 
and D Center at Wisconsin is the largest agency 1n 

the Midwest with educational research and develop- 
ment as its primary function. 


The informal training of educational dee 
and developers to supplement the formal PME nd 
in the various instructional departments S thi Gars 
leges, has been a significant contribution oe Е а 
ter to improving education through тавак " e 
velopment. For example, during 1966-67, infor ma’ 
training was provided for four postdoctoral fellows, 
sixty-three graduate students employed part time 
by the Center, ten ESEA Title IV doctoral fellows, 
ten NSF fellows, five NDEA fellows, several Univer- 
sity fellows, and forty-two learning specialists em- 
ployed by the public schools. Twenty-eight persons 
affiliated with the Center were awarded PhD’s by 
the end of 1967 and are located throughout the coun- 
try in a variety of leadership positions, A large group 
of persons has been awarded the Master’s degree also. 


THE WISCONSIN RESEARCH AND DEVELOPMENT CENTER FOR COGNITIVE LEARNING 153 


TECHNICAL SUPPORT 


The primary components of the Technical Section 
are a measurement and design group; production 
specialists; and computer experts. The design 
group provides assistance to the research personnel 
on matters of research design and evaluation. 
Equally important, this group is continuously devel- 
oping new designs to meet the needs of the Center 
and refining statistical procedures for the analysis 
of data. A measurement expert provides assistance 
on problems of measurement and develops more ef- 
ficient measurement tools and procedures. Comput- 
er specialists coordinate data analysis, from the 
conversion of raw data to computer compatible form 
through the presentation of the processed data in 
tables and graphs, and assist in the application 
of computer technology to educational research and 
development. A curriculum specialist assists the 
staff in identifying and using appropriate reference 
materials and in the production of graphical materi- 
als. Production experts assist with the development 
of instructional, experimental, and informative ma- 
terials in the various media. 


STAFF 


Execution of the Center programs requires 
proper interdisciplinary staffing. This Center sub- 
Scribes to the philosophy that the most productive 
research and the development of effective instruc- 
tional systems will derive from teams composed 
primarily of behavioral scientists, subject-matter 
specialists, and experts in curriculum and other 
areas closely related to educational practice. The 
behavioral scientist contributes his research skills 
and knowledge of fundamental processes in cognitive 
learning; the subject- matter specialist provides 
knowledge of the subject field; the experts in curric- 
ulum and other fields contribute knowledge about 
methodology and other relevant instructional varia- 
bles. 


Studies of cognitive learning, included in Pro- 
gram 1, are headed by behavioral scientists con- 
cerned primarily with synthesizing and advancing 
knowledge in this area, The other experts contrib- 
ute to these studies as team members when cogni- 
tive processes related to specific subject content 
are studied, In Program 2 in which processesof in- 
struction are studied and instructional programs 
are developed, the projects are headed by subject- 
matter or curriculum specialists concerned with de- 
veloping an instructional system in a subject area; 
however, a behavioral scientist is a member of the 
team. Experts in school organization and the soci- 
ology of groups are involved with other staff in the 
development and maintenance of the facilitative en- 
vironments, Program 3. Communication special- 
ists carry out the dissemination program. 


This mutually beneficial involvement of the staff 
has two natural consequences. First, the basic 
Studies in learning are concerned with developing 
knowledge that is relevant to educational practices. 
Second, the Studies of instructional processes and 


In addition to the Center staff, many outstanding 
interdisciplinary groups are available within the Uni- 
versity for cooperative planning and execution of ac- 
tivities. Personnel from three of the outstanding ac- 
tivities in The University of Wisconsin are continu- 
ously engaged in consultation and team work with 
the Center staff: WHA-TV, a foremost producer of 
educational TV programs in the nation; the Survey 
Research Laboratory, a large and distinguished in- 
terdisciplinary survey team; and the Computing Cen- 
ter, one of the best in the nation. Thus, the Wiscon- 
sin R and D Center does not require a large perma- 
nent staff in each of these areas of specialty. Rather, 
programs and activities are planned and executed 
cooperatively as part of the total University of Wis- 
consin contribution to improvement of education 
through research and development. Other relevant 
interdisciplinary centers and laboratories at the Uni- 
versity include the Center for Demography and Ecol- 
ogy, Mass Communications Research Center, and a 
Social System Research Institute, 


Twenty-two persons of professorial rank were af- 
filiated with the Center, 1967-68. The departments 
in which they held rank included Computer Science, 
Curriculum and Instruction, Educational Administra- 
tion, Educational Policy Studies, Educational Psy- 
chology, English, Psychology, Sociology, and 
Speech. Some professors held rank in two depart- 
ments. 


ADMINISTRATION 


Administratively, the Center is located in the Uni- 
versity’s School of Education but draws upon the en- 
tire University for human and other resources. The 
working relationship between the Center and the Uni- 
versity is manifested in three ways: professors in 
various University departments are members of the 
Center's Faculty; both the Dean of Education and the 
Chancellor of the University are involved in review- 
ing the Center's activities; and University officials 
Serve on the Center's Executive Committe e, the 
Policy Review Board, and the Advisory Council. 
Figure 4 outlines the administration and ma nage- 
ment of the Center. 


The Center Director is responsible for planning 
the total Center program and its broad Strategies, 
A director of the research and development pr o - 
grams, a director of dissemination, a director of 
operations and business, and a director of a techni- 
cal section, with the Center Director, comprise a 


Management Council. 


The Executive Committee makes recommenda- 
tions regarding policies, programs, and budget, It 
consists of the Center Director, who is also the 
chairman, three members elected by the Principal 
Investigators, and three members appointed by the 
Dean of the School of Education. 


A Faculty of Principal Investi; ators, c 
of professors in various University de 
participate in the Center's Program d 
and they plan and execute the Center's p 


onsisting 
partments, 
evelopment, 
rojects. 


The Policy Review Board brin 


the instructional products that are developed in Pro- 
у Sram 2 have a sound foundation in the behavioral sci- 
ences and the subject disciplines. 
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Figure 4: Wisconsin Research and Development Center for Cognitive Learning organization chart 
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Program 1: Processes and Conditions 
of Learning 
Processes and Programs of 
Instruction 


Facilitative Environments 


External 


Program 2: Contacts 


\ Program 3: R & D Support 


Faculty of Principal Invest 


Board is composed of representatives from the of- 
fice of the Chancellor of the Madison Campus; the 
deans of the School of Education and the College of 
Letters and Science; the Director of the Wisconsin 
Improvement Program; the Superintendent of State 
Department of Public Instruction; a member of the 
Wisconsin Coordinating Committee for Higher Edu- 
cation; the Executive Secretary of the Wisconsin Ed- 
ucation Association; three departmental chairmen of 
the College of Letters and Science and the School of 
Education; and the Center Director. The Board an- 
nually reviews the Center's mission, organization, 


Programs and staffing; recommends overall policies; 
suggests funding levels, 


gr: 


A National Evaluation Committee confers annual- 
ly with Center personnel on the Center’s progress. 
The Committee consists of nine individuals of nation- 
al reputation interested in the problem area of the 
Center and committed to the improvement of educa- 
tion through research and development activities. 


teen school 


Individuals of six- 
schools and 


including public and parochial 
al institute, are included. 


systems, 
a technic: 
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LOCUS OF OPERATION 


Because the R and D Center is effectively inte- 
ated into the University complex, both on an ad- 


ministrative and disciplinary level, it is natural 
that the Center program incorporates the philosophy 
of The University of Wisconsin regarding the im- 
provement of society through research, develop- 
ment, and training. This philosophy is best ex- 
pressed in a widely quoted statement: 


Whatever may be the limitations which 
trammel inquiry elsewhere, we believe that 
the great state University of Wisconsin should 
ever encourage that continual and fearless 
sifting and winnowing by which alone the truth 
can be found. (Taken from a report of the 
Board of Regents in 1894.) 


President Harrington has from the beginning im- 


plied the extension of these concepts to educational 
research through the R and D Center, and he contin- 
ues to give generously of his time to assist the Cen- 
ter in achieving its goals, and to assure a climate 
in which the Center may operate more effectively. 


The Madison Campus of The University of Wis- 
consin is the foremost research agency in the state t 
and is among the leading research-oriented univer- 
sities in the world. The commitment to educational 


THE WISCONSIN RESEARCH AND DEVELOPMENT CENTER FOR COGNITIVE LEARNING 155 


research is substantial as may be inferred from a 
statement by then Chancellor Robben Fleming on 
July 1, 1965: 


With so much emphasis on education in to- 
day's world, it would be easy to drift along the 
well-marked paths of the past. The strength 
of our School of Education lies in its restless 
innovation. In cooperation with the schools of 
the state, it is forever trying new ways to im- 
prove the primary and secondary schools. In 
residence its curriculum is marked by an un- 
usual degree of cross-fertilization, and by an 
interdisciplinary faculty. Itsresearch is 
probing the frontiers of the learning process, 
asking, in effect, whether we are on the right 
path to begin with. 


Implementation of the philosophy implied in the 
preceding statements as applied to the Rand D Cen- 
ter has been demonstrated in two notable ways. 
First, the Policy Review Board of the Center, the 
most powerful governing agency of the R and D Cen- 
ter, is chaired by James Cleary, Vice-Chancellor 
of Academic Affairs. This Board reviews the pro- 
gram of the Center and takes major responsibility 
for assuring adequate staffing of the Center in light 
of its program. The second example of implemen- 
tation is that the actual state support, not an indi- 
rect contribution, to the Center has increased from 
approximately $10, 000 in 1965-66 to $108, 000 in 
1967-68. This state contribution is used almost to- 
tally to pay the salaries of principal investigatorsof 
the Center — professors who hold appointments inan 
academic department of the University and are budg- 
eted fifty percent or more in the Center. 


The School of Education is committed to the phi- 
losophy that the continuous improvement of public 
School education requires unceasing research and 
related activities. Dean McCarty, shortly after ar- 
rival at the University in 1966, indicated that the 
R and D concept is the most significant educational 
innovation in recent decades and that all for ms of 
worthy educational research merit support. Three 
of many possible examples point to the high priori- 
ties given to educational research and related activ- 
ities at The University of Wisconsin. First, 
$4,024,298 was expended on educational re- 
search and program experimentation in 1965 through 
the Schoolof Education. The four sources of the 
support were Federal agencies, the State of Wiscon- 
sin, private organizations, and local school systems. 
Second, in the first seven years of the Cooperative 
Research Program of USOE, the University was 
awarded grants for thirty-three projects. Third, 
the amount of state support for research tothe 
School of Education increased from $294, 963 in 
1964-65 to $804, 976 in 1966-67. 


The interdisciplinary nature of the School of Ed- 
ucation is a major factor in the eminence of the Uni- 


Versity in educational research and related activities. 


In 1965 the membership in the School of Education 
was 1,075 professors {from seventy-two departments 
of the University. Graduate work in preparing edu- 
cational researchers is interdisciplinary. Profes- 
sors in many departments of the University offer 
courses taken by graduate students pursuing pro- 


grams to prepare for educational research. Further, 
many PhD candidates in the various departments of 
the School of Education pursue minors in depart- 
ments of the College of Letters and Science while 
many PhD candidates in Letters and Science pursue 
minors in departments of the School of Education, 


SOURCE AND STABILITY 
OF FUNDING 


Any organization requires stable funding over a 
period of years in order to secure excellent staff at 
reasonable salary levels. Stability of funding was 
an important consideration in starting the Center. 
This Center, in line with the general USOE policy, 
was awarded a 5-year grant, starting September 1, 
1964, by the USOE with three important provisions: 
First, that the contract would be amended annually; 
second, that, if for any reason, itbecame necessary 
to discontinue the Center the phasing out process 
would be accomplished over a period of time mutually 
satisfactory to The University of Wisconsin and 
USOE; and third, in the fourth year, the achieve- 
ments and operations of the Center would be evalu- 
ated comprehensively in order to arrive at a dec i- 
Sion regarding a second 5-year period. The Center 
was awarded approximately $500,000 for thefirst 
year of operation. Each amended contract for the 
next three years showed a modest increase in the 
level of support. 


The second source of support is the State of Wis- 
consin through The University of Wisconsin. As 
noted earlier, some monies are made directly avail- 
able to the Center. This is, however, not the main 
University support. The largest support lies in the 
availability of personnel to the Center – administra- 
tors, professors, graduate students, and classified 
personnel — and also other personnel in specialized 
fields such as computer science, opinion sur vey, 
TV production, and equipment development. The 
University also has leased a facility and provides 
the many services that are necessary to make an 
operation function well. 


Although not a direct source of funds to support 
Center activities, the staff of four local school Sys- 
tems, particularly the subject- matter consultants, 
the building principals, and unit leaders, give con- 
siderable time to refining the concept of individually 
guided education in multiunit schools. Also, ideas 
and innovations of the local schools and the R and D 
Center are demonstrated through Title III projects 
in local schools. The State Department of Public In- 
struction is the principal disseminator of Rand D 
field-tested ideas and materials in Wisconsin. 


CONCLUDING STATEMENT 


The preceding description of the Wise i - 
search and Development Center for Coenie алсы. 
ing was written late in the third year of the Center’ 
existence. It is anticipated that there will not a 
sharp changes during the next decade inany of th 
eight major components: problem area, outcome ч 
sought, research and development pro g rams S 
strategies, dissemination program and strategi 206 
pattern of staffing, pattern of administration end 
management, locus of operations, and Source d 


i 
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Stability of funding. Though Sharp changes are not 
expected, continuous improvement in allofthese 
areas is encouraged. Annually the Executive Com- 
mittee of the Center evaluates each program and 
project and the related strategies, The National 
Evaluation Committee gives particular attention to 
the significance of the problems being attacked and 
the adequacy of the Strategies being employed. The 
University Policy Review Board examines all ele- 
ments of the Center activities but gives particular at- 
tention to proper Staffing. Field visitors appointed 
by USOE officials, and USOE officials also evaluate 
all components of the Center. University officials 
as individuals, consultants invited by the Center 
Staff, and school people provide evaluative informa- 
tion. The information from all sources is carefully 
analyzed by all the Center staff, but particularly by 
the Executive Committee, Management Council, and 
Director. Annually, the information is formally put 


into program plans for the ensuing year. Changes 
are also made as essential at any time during the 
year. When any element does not function properly 
Or produce at a reasonably high level, it is modified, 
The Center now demonstrates hi gher productivity 
and more sophisticated Strategies than a year ago. 
Its improvement mechanisms are designed to accel- 
erate these trends. 


FOOTNOTE 


1. Miss Elizabeth Schwenn, Project Assistant, as- 
sisted in editing this chapter. The Executive 
Committee and Management Council of the Cen- 
ter contributed to the program description of 
the Center and the various faculty and their 
research and development staff are responsi- 
ble for the results as reported herein, 


— An Output-Oriented Model 
of Research and Develop- 
ment and Its Relationship 
to Educational Improvement 


FOR SEVERAL months I have followed 
the pages of Science attending to the continuing dis- 
cussions of research and science policy, generally 
concerning myself with the degree to which the pol- 
icy questions which arise in educational research 
and development can, must, or ought to becon- 
sidered as a subset of those which arise for science 
as а whole, Thus it has been with no small degree 
of interest that I have read the articles prepared by 
Philip Handler (1967), George Daniels (1967), Har- 
vey Brooks (1967), Lee DuBridge (1967), and Mi- 
chael Reagan (1967). 


Coincidental to this continuing dialogue I have 
been for the past 2 years engaged in a general ef- 
fort to improve the quality, effectiveness, and im- 
pact of the total research effort in education. No 
small part of that time has been spent grappling in 
practice with many of the same kinds of issues as 
receive explication in the articles cited above. For 
some months I have been thinking about research 
and development in education trying to develop a 
model which would express the different functions 
within the total research effort, the various sources 
of initiative for these different kindsof activity, and 
the relationships among both the functions and the 
Sources of initiative, My thinking has, of course, 
been in no small measure stimulated by recent de- 
bate about the ways in which schools, instruction, 
and education are likely to be improved most quick- 
ly and with the most substantial cumulative and last- 
ing impact, 


Reviewing the several statements cited above in 
terms of their general usefulness for spotlighting 
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and clarifying analogous problems in the field of ed- 
ucational research revealed the particular value of 
Reagan's analysis as a summary of points of view 
already advanced and as an identification of the gen- 
erally individualistic starting point of many of the 
theoreticians of science and research policy. The 
model presented in this article constitutes, I think, 
an instance of exactly that for which Reagan called 
(1967), namely, a discussion of research and devel- 
opment as seen and understood from the point of 
view of someone who sees himself as a Sponsor or 
user of research and developmentto achieve certain 
broad purposes in a particular field of human activ- 
ity. It was not developed directly in res ponse to 
Reagan's suggestion; in fact, it was only after a col- 
league recalled his article with Specific reference 
to the similarity of view on the research and devel- 
opment distinction that I realized how closely what 
I had formulated came to responding to Re agan's 
Suggestion for more dialogue on the part of users 
of research to more adequately balance that Which 
had been generated by the performers, 


The Guba-Clark Model 


Typically, the context for the debate about re- 
search and development for education isadiscussion 
of the change process in education. Various models 
of change have been proposed. Theone Which seems 
to have the highest degree of Currency in education 
at the moment places research, development and 
dissemination in a linear arrangement beginning on 
the left with inquiry (research) and Proceeding to th 
right through development, diffusion, and adoptio e 
(see Figure 1). In proposing the model Egon Cubs 
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and David Clark (1965) called three caveats to atten- 
tion. First, they noted that the modelwasconstruc- 
ted on logical grounds and that it was largely unsup- 
ported by empirical research. Second, they pointed 
out that it was not necessary for change to begin at 
the research or inquiry stage. Third, they noted 
that the model itself was a uni-dimensionalanalysis 
of change roles which are influenced by a multi-di- 
mensional range of variables not entirely accommo- 
dated by the model's structure. 


As a model of the change process, this particu- 
lar schema is simple and logical. However, those 
of us who have worked intensely on problems of re- 
search policy in education see some shortcomings. 
The Guba-Clark model does not emphasize suf- 
ficiently within its structure that initiative for action 
of quite different kinds can take place at any point 
intheir continuum. Because of its linear nature, 
and despite the second caveat mentioned above, the 
model unwittingly implies that innovations begin with 
the findings generated by fundamental research. 


The Output Model 


The purpose of developing an alternative model 
is to create a heuristic which a) illustrates the es- 
sential differences between research and develop- 
ment activities and b) shows how the two are - or 
can be — related to one another and to the operating 
educational system. Sucha model ought to illustrate 
the different sources of initiative and motivation for 
beginning various activities. It should be able to 
show or imply the interplay among all the functions 
in the effort to improve instruction, 


The model developed is based on the conviction 
that research, development, and school operations 
are different kinds of activities with quite different 
objectives or outputs. It indicates that initiatives 
for each kind of activity are the results of decisions 
based on quite different kinds of data and equally dif- 
ferent kinds of internal and external needs. T he 
model implies that while there may be a strong log- 
ical flow from the production of knowledge through 
the development of processes to their installation in 
Operational settings, there may be just as strong a 
flow backwards as operational problems define de - 
velopment programs, which, in turn, reveal the 
need for certain basic information and theory. 


Figure 2 expresses the model, Three planes are 
shown, each symbolizing the different orientation of 
activities conducted under research, development, 
and school operations. (The model is - as are all 
models – an abstraction from reality. In the real 
world these activities are not always neatly sepa- 
rated either in time or location. The point of con- 
Ceptually – and therefore graphically - separating 
them here is to illustrate the essentially different 
Orientations of the three types of activity and the 
Consequences of those differences.) For each ac- 
tivity represented in Figure 2, the model depicts an 


initiative leading to an output characteristic of that 
activity, 


tation оне Plane symbolizes the knowledge orien- 
generate йы оу The object of research is to 
tures of re knowledge, One of the significant fea- 

Search is that when an activity is begun 


EE 


the specific outcome is not known. For research, 
C represents an initiative undertaken which culmi- 
nates in a research finding represented by Fc. 


The middle plane symbolizes what I call the proc- 
ess orientation of development. The object of devel- 
opment is to produce materials, techniques, proces- 
Ses, hardware, and organizational formats for in- 
struction which accomplish certain pre-specified 
objectives construed to be part of the broader goals 
of education. One of the signficant features of de- 
velopment is that when an activity is begun, the ob- 
jective is known or established at the outset, The 
objectives for a development project are cast in the 
form of performance specifications (PS), and all ac- 
tivities are geared to producing the necessary prod- 
ucts and processes which will meet those specifica- 
tions. In Figure 2, B indicates an initiative under- 
taken for development culminating in the creation of 
а process which meets performance specifications 
PSp. 


The top plane symbolizes the activities character- 
istic of school operations. The operating education- 
al system is production oriented, Thus, the object 
of school operations is to act upon human beings in 
order to train and develop in them various skills, 
attitudes, beliefs, and knowledge calculated to serve 
both society and themselves. Certainly one of the 
significant features of initiatives in school operations 


Figure 2. An output model of educational research 
and development 
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is the weight of the responsibility on the school ad- 
ministrator for choosing the right kinds of proces- 
ses to achieve the outputs that society and individu- 
als specify. In Figure 2, A represents an initiative 
to install a process leading to the production of ed- 
ucation output EO}. 


А '"Walk- Through'' of the Model 


To illustrate the relationships among the three 
types of activities, consider the following example. 
A responsible school Official, faced with evidence 
that certain outputs desired by the society are not 
being achieved for a significant portion of the chil- 
dren in his charge, searches other school operations 
and ongoing or completed development projects for 
processes designed to meet his need. Should he find 
nothing to suit his particular problem (e.g. , the low 
reading achievement of culturally disadvantaged chil- 
dren), he may then exercise his prerogative to call 
for the initiation of a project to design and develop 
à process whose performance specificationsare 
Such that upon installation of the process in his 
School, it will yield the desired educational output 
(e.g. , increased level of reading achievement in the 
target population). 


Once the initiative for the development project 
has been undertaken and the performance specifica- 
tions established, the development project then con- 
ducts a search for relevant research findings which 
may offer clues to guide the development project. 


Figure 3. А ‘‘Walk-Through”’ of the model 


(Whether or not this step is taken after the project 
is begun or immediately before it is not really im- 
portant. What is crucial is that at some point near 
the very beginning of the effort such a search is 
made.) Impressed with a particular finding (e. g., 
the tremendous impact of parentalattitude on student 
achievement as revealed by the survey conducted as 
part of the Plowden study), the project m ay decide 
to develop a process which deliberately tries to en- 
gender a large measure of parental involvement in 
home instructional experiences carefully geared to 
complementary experiences in the school setting. 
Having made that decision the developers may then 
discover that they require further information about 
the specific nature of optimum parent-child interac- 
tions to stimulate maximum learner achievement. 
They may therefore call for a specific initiative of 
a research activity to generate further data to guide 
the development of materials. When useful findings 
are identified they can be incorporated in the devel- 
opment effort which then proceeds to successfulcon- 
clusion. When, using iterative techniques of design, 
development, trial, and redesign onthe basis of 
feedback, materials encompassing both school ex- 
periences and parent-child interactions in the home 
are successfully developed and validated, they may 
then be transferred to the operating setting where 
the administrator may install the m as part of his 
instructional program. 


This example is illustrated in termsof the model 
by Figure 3. ЕО; at No. 1 symbolizes the social 
demand for a certain kind of educational output (in 
the example just given, higher reading achievement 
for culturally disadvantaged children). This demand 
creates pressure on the school administrator to re- 
spond with some sort of initiative, That initiative 
is represented by A at No. 2. It symbolizes his 
search for an effective process to install. Since he 
did not find it, his response was to call for a devel- 
opment initiative (B at No. 3). The next step was 
to develop the performance specifications (PSy at 
No. 4) such that they correspond to the educational 
output desired by society. Once the specifications 
for the development project are established, the next 
Step is to survey related research seeking guidance 
for the development effort. The searchis conducted 
and the finding (Frx) of relevance to the perform- 
ance specifications and the desired educational out- 
put (e.g., the significance of parental attitude) is 
incorporated into the development project (No. 5). 
The call for additional research assistance is sym- 

bolized by D at No. 6 and the incorporation of rele- 
vant findings (again Егу) from that initiative into the 
development effort is symbolized by the solid arrow 
to the development line at No. 7. Number 8 repre- 
sents the completion of the development project, No, 
9 the incorporation of the process into school oper- 
ations, and No. 10 the production of the desired out- 


Put, higher levels of reading achievement, as а con- 
Sequence, 


A Sampling of Other Possible Interactions 


The depiction of a sampling of other i i 
" ч = 
teractions among research, development. а $ a 
ations can be found in Figure 4, И в 


Example: A school 


official feels the need to as- 
Sess the degree to which instructional programs a 
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Figure 4. A samplingofother possible interactions 


Serving a particular target population. He calls 
therefore for an initiative in research. This is rep- 
resented by the D/D/F-F interaction. 


Example: An organization engaged in develo p- 
ment independently concludes that it would be useful 
to develop a certain process or product for instruc- 
tion. This is represented by the B/PSp interaction. 


Example: Research is initiated for its own sake 
and pursued solely for the knowledge which it pro- 
duces. No findings have yet been incorporated 
either in development or operations. This is sym- 
bolized by the C/F-F-F relation in research. 


Example: Research initiated for its own sake 
yields the findings that certain organizational struc- 
tures for large city school systems are always prob- 
lematical or that a certain vitamin supplement ad - 
ministered between the ages of 5 and 7 can prevent 
а form of mental retardation whose appearance can- 
Dot be detected until somewhat later. Neither one 
of these findings needs to pass through development. 
Each can be implemented directly in school opera- 
tions (if so desired), This kind of relationshipof re- 


3. 8rch to practice is illustrated by the G/ F-F/H/EQg 
interaction, 


form of 01е: _Finally, consider an instance in the 
Tesearch to linear flow or Guba-Clark model from 


? development to implementation. Re- 


search on early childhood and cognitive growth, for 
example, uncovers a number of f indi ngs some of 
which may suggest the development of processes 
and environments which can actively enhance such 
growth. Development efforts are consequently sup- 
ported and carried through to completion. Theavail- 
ability of the resulting products and processes is 
made known to educational administrators whoare 
thereby persuaded to incorporate the new ly devel- 
oped early childhood instructional programs,at 
either private or public expense, into the Nation's 
educational system. This kind of Sequence is rep- 
resented by the J/F-F-F/K/PSj/L/EOj interaction, 2 


THE MODEL AS AN HEURISTIC 


All of these representations in Figures 2, 3, and 
4 are fairly obvious and straightforward. Theu se 
of the model as an heuristic, however, profits from 
further explanation. One of these uses pertains di- 
rectly to the problem of ‘‘change process" as ap- 
plied to education. The modelis structured to illus- 
trate that the incorporation of research findings in- 
to development is just as important and oftentimes 
as difficult a proposition as incorporating newly de- 
veloped processes into operational settings. The no- 
tion that there are obligations on both research and 
development to transfer their “products” to other 
activities means that each must pay carefulattention 
to the way in which its outputs are presented and Я 
perhaps the very way in which the outputs are pro- 
duced. In other words, the requirement that even- 
tually there be transfer or incorporation into anoth- 
er type of activity places constraints upon the pro- 
fessional behavior in each activity which cannot be 
ignored without compromising later impact. 


This requirement is particularly true for devel- 
opment projects, but it is as true for research ac- 
tivities. A few concrete examples illustrate the 
point. One such example is the researcher who pub- 
lishes his findings in a sloppy or difficult for mat 
and thereby hinders the likelihood of their being in- 
corporated ultimately into practice. The research- 
er who inadvertently conceals or com promises 


his methodology or design encounters similar prob- 
lems. 


Similarly, the final requirement for a develop- 
ment project is that it be usable in Operational set- 
tings. The ultimate desire to incorporate the devel- 
oped process in school operations means that one of 
the performance specifications for dev elopment 
must always be the provision of procedures (teach- 
ers' manuals, training procedures, etc. )for accom- 
plishing the installation of the innovations. Ifthe de- 
velopment is undertaken without reference to that 
fact (if, in short, the requirement for transfer is 
not built into the performance specifications), the 
developer may well have rendered his product un - 
usable. Hence, for example, the desire to involve 
teachers and other practitioners in the development 
process stems from the need to have their expertise 
and experience continually represented. They con- 
stitute, in effect, the embodiment of many of the op- 
erational constraints and possibilities in schools with- 
in which the finished process willoperate, There are 
other good reasonsfor involving teachers as well, not 

the least of them being the respectability lent to the 
project in the eyes of the practitioners at large by 
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virtue of the meaningíul presence oí teachers in the 
effort. This last consideration is of no small impor- 
tance in securing acceptance of the innovation in the 
profession at large. The nature of its importance, 
however, should not be mistaken; the involvement 
must be meaningful and not merely window-dressing, 
for the respectability is lent by their presence only 
if their contributions are fully utilized andtheir real 
knowledge of what is or may be possible is thoroughly 
weighed in the development effort, 


One final point might be made about the model’s 
portrayal of the possibility for transfers back and 
forth among research, development, and operations. 
That is the obvious emphasis which needs to be giv- 
en to the problem of information flow and the need 
for carefully considering techniques for installation 
of better knowledge and better processes into their 
intended settings. Only part of this is the direct 
responsibility of the researchers and the developers. 
Those with obligations for considering the entire 
R and D system for improvement need also to direct 
their attention to the diffusion process. 


A second feature of the model as an heuristic is 
the way it helps to clear up part of the problem of 
distinguishing between basic and applied research 
in education.? The model clearly implies that basic 
research (studies generated independently inre- 
search for the sake of the findings alone) and ap- 
plied research (research conducted to serve a par- 
ticular need identified by people engaged in develop- 
ment or operations) differ from one another primar- 
ily in terms of the intent of the initiator.* Thus the 
knowledge-orientation of the basic researcher is 
central to his activity. Applied research is also 
supported for the knowledge which results from it, 
but the initiator of the research knows to what in- 
Strumental use he is going to put his findings. By 
depicting both applied and basic research as similar 
kinds of activities, the model implies that in and of 
themselves they look very much alike. The proce- 
dures, the design, the sophistication must all be on 
par if either is to be valuable. What distinguishes 
the two from one another, if anything does, are the 
purposes for which they are initiated, 5 


A third feature of the model as an heuristic is its 
Suggestion that decisions 
each of the three 
different criteria 
ple. The fundam 


ularly diffic 


becomes a partic- 
ly decisions i 


the kinds of lone- 


data and statements of need when planning future 
activities. 


Finally, the frank attempt to represent each of 
the activities in terms of particular kinds of outputs 
may well be the most significant aspect of the model. 
It forces the viewer of the model to consider what 
the outputs of each activity are and to think about 
how the outputs of each activity are of use to one an- 
other. The outputs of research, for example, are 
knowledge. Some oí the knowledge produced through 
research will find its way into development and into 
school operation. Are there ways of improving the 
output of research, making it more powerful, in- 
crease the likelihood of its being of use to instruc- 
tion and education? 


What about the outputs of development? They con- 
stitute, on the one hand, the validation of research 
and, on the other, the means by which the educa- 
tional system can improve the manner in which it 
carries out its functions. How can development be 
improved, how can research be organized to be of 
greater use to development, and how can the educa- 
tional system itself orient its organization to the re- 
curring need for the installation of more powerful 
validated techniques? 


Finally, what happens to educational operations 
when they begin to view their responsibilities 
in terms of output? The contrast can perhaps be 
most sharply drawn by considering the implications 
of the notion of grading schools on the basisof their 
outputs rather than students on the basis of their 
performance, The existing practice of grading stu- 
dents assumes at bottom that the student is respon- 
sible for his learning and that his failure or success 
is a tribute or a consequence of factors intrinsic to 
him. The idea of grading a school on the basis of 
its outputs assumes quite to the contrary that all 
Students can learn and that the responsibility of the 
schools is to make that happen. (We do not, for ex- 
ample, judge the success or failure of medicine or 
law by the patient's or client's end state; we judge 
it by the degree to which the doctors or lawyers 
skillfully utilized the most sophisticated practices 
in attempting to serve the client. We certainly do 

not *'grade'' the patient or client; quite to the con- 
trary, it is the professional services themselves 
which are assessed. An output orientation for 
School operations would cause the same reorienta- 
tion of the direction of assessment in education. ) If 
the schools themselves are judged in terms of the 
degree to which they are accomplishing their ''pro- 
duction goals,'' increasingly they may come to one 
ent their activities to assessing their own effective- 
ness, identifying the techniques and processes which 
need improvement and, as a consequence, calling 
with increasing sophistication for the kind of devel- 
opment activity and research support which will 
provide the basis for continuing improvement. 


FOOTNOTES 


1. The author is Director of the Program Planning 


ue Evaluation Staff of the Bureauof Research, 
Der Office of Education. This paper is writ- 
n in Gideonse’ private capacity. No official 
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support or endorsement by the U.S. Office of 
Education is intended or should be inferred. 


2. This example is, of course, clearly an ideal con- 
ceptualization. It is instructive to kee p the 
model in mind as one looks back over the past 
four years at the tremendous developing inter- 
est in the establishment of early childhood 
educational programs. While it is certainly 
difficult if not impossible to establish an indi- 
vidual cause for such a complex phenomenon, 
it is nonetheless significant, I think, that Ben- 
jamin Bloom published his volume Stability 
and Change in Human Characteristics (1964) 
just at the time the Office of Economic Oppor- 
tunity was beginning its planning toward the 
development of programs tofight the war 
against poverty. The significant thing about 
Bloom's volume, however, was that while his 
conclusions firmly underscored the impor- 
tance of the early years in the development of 
cognitive skills, he also pointedly observed 
that there had been little actual experimenta- 
tion designed to create environments to en- 
hance such skill development. I do not think 
that it does violence to Bloom's argument to 
interpret it as a call for rigorous development 
efforts designed to produce environments and 
instructional programs which have the effect 
of enhancing human capabilities. The prob- 
lem, however, is that which exists in all so- 
cialdomains. There are large numbers of 
children now whom we cannot afford to ignore, 
and the consequence has been the attempt to 
create operating early childhood programs 
based on those convincing research findings 
without first having gone through a develop- 
mental stage. The result has been а some- 
what marginal impact on the target population 
despite the clear implications of the research 
summarized by Bloom. 


3. I think it is also useful at this point to recall 
Reagan's view that the social sciences present 
something of a different picture in regard to 
the distinction between basic and applied re- 
search. As he points out (Reagan, 1967, 
p. 1385), no matter how abstruse and abstract 
its practitioners attempt to become, social 
Science research is inherently related to 
potential applications. 


4. The initiator and performer are, of course, not 
necessarily one and the same person or insti- 
tution. The actual initiator of the research 
project may be a School man, a developer, a 
researcher, or a research administrator. The 
performer of that research effort may or may 
not have the same ultimate purpose as the ini- 
tiator in mind as he undertakes the activity. 
Thus, for example, it would be perfectly pos- 
sible for a research administrator to stimu- 
late a series of research activities relating to 


reading which he views as applied research 
necessary for a development effort to build 
improved reading curricula, whilethe per- 
former of that research sees it as a basic re- 
search effort in perceptionor the psychomotor 
skills associated with reading. 


5. This view, I believe, fits fairly well with one 
part of Harvey Brooks’ analysis of the distinc- 
tions between basic and applied research 
(Brooks, 1967, p. 1706) when he noted that 
“аѕ definite categories, (the terms) basic and 
applied tend to be meaningless, but as posi- 
tions on a scale within a given environment 
they probably do have so me significance. ” 
The principal shortcoming with Brook's anal- 
ysis, however, is that it proceeds almost en- 
tirely on the presumption that the distinction 
can be resolved by approaching it in some way 
from the researcher's point of view. My ex- 
perience in the administrationof research, as 
the model presented in this paper clearly in- 
dicates, leads me to believethat the research- 
er's view is only one of several which must 
be taken into account in attempting any analy- 
sis of the distinction between basic and applied 
research. The presence of rather different 
criteria and alternative vantage points convin- 
ces me that Reagan's argument for abandoning 
the distinction within research is much more 
persuasive as is the suggestion that the criti- 
cal categorization is that between research 
and development. 
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ABSTRACT 


This study wasaninvestigation of the effectiveness of eight types of visual illustrations used to complement 
oral instruction. An attempt was made to evaluate the instructional value of black and white and colored visual 
illustrations. Two hundred sixty-four eleventh-grade students were randomly assigned by class to one of nine 
treatment groups. Student learning was evaluated by means of five criterion tests designed to measure differ- 
ent educational objectives. Comparisons among the means of the nine treatment groups yielded significant dif- 
ferences. It was found that the realism continuum for still visuals is not a reliable predictor of learning ef f i- 


ciency; however, visual illustrations were found to significantly increase student learning on three criterion 
tests. Color in visual illustrations did not appear to be an important variable in increasing student learning in 
this study. 


RECENT RESEARCH and literature indicate that 


tween the visual and the object or situation to which 
too much detail in an illustration tends to reduce 


it refers, the easier it will be for the students to 


student learning [ Broadbent (3, 4), Attneave (2), learn from it. 
Jacobson (15), Travers (24, 25), and Dwyer (9)] 
and that detail in visual illustrations m ay be re- REALISM THEORIES 


duced so much that interpretation of the illustra- 
tions by students may cause unpredictable responses 


The present study evolved out of the inconsisten- 
[Fleming (11)]. Apparently, there is a conflict be- 


cies resulting from research guided by the **estab- 
tween the visual realism continuum and results of lished’’ theoretical orientations, specifical ly the 
recent research. Allen (1), Hoban (14), Schramm iconicity theory identified by Morris (19), the sign 
(22), and Dwyer (9) have stated that extensive re- 


similarity orientation developed by Carpenter (6), 


search should be conducted on the physical charac- and Dale's (7) cone of experience. For purposes of 


teristics inherent within pictorialillustrations which 
lead to increased learning and to the attainment of 
specific educational objectives. 


Figure 1. Realism Continuum for Still Visuals 

VISUAL REALISM CONTINUUM ws 

A visual is said to be more realistic on the basis = = Э = 
of an increase in the quantity of physical character- E 8 8 5 Nw 
istics the visual has in common with the object or B © m o в 
situation. The visual realism continuum is founded M RT NE Ош с 
on the contention that as the amount of realistic de- в а E 2 3 E > = 
tail in the visual illustration is increased, learning 2 3 ЕІ E m oO BS 
is also increased, and, thus, the visual realism я 9 as л n PAP 
continuum extends from the object or situation itself CEN] з cv E * а э 
to а very simplified abstract line representation. oo 8 S £ 8 2 а 
(See Figure 1.) A visual considered to be 100 per- a E yy ; P е в E] 
cent realistic with respect to a designated object È È i E E SG UR & 
could not be differentiated from the object because а c w S 8 om mg 
it would possess exactly the samequalities possessed S 3 з 3 = ©з 9 
by theoriginal object. Oral instruction, there- В э dd wm 7B Б hh ЖЮ 
fore, would contain a low degree of realism since ee S 2 2 - = 2 E 
it would not physically represent the denota, Conse- d 2 8 E ез 3 8 
quently, it would follow that visual illustrations, о 3 a B OX © E © E 
Eig from colored photographs to the black and SAG ш e ц 
M ite abstract line representations, may vary in 
their ability to 


educati '! promote student learning of specific Low Efficiency in Facilitating Learning 
Si ‘onal objectives. Research based u pon the 

js Tealism continuum has been guided by the hy- 
Pothesis that the higher the degree of similarity be- 


High 
Points on the Realism 


Continuum in acco: 
the assumptions predic 


rdance wi 
ted by the “Realis tog 
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convenience the above orientations are referred to 
collectively as the realism theories. According to 
Finn (10), the basic concept around which audiovis- 
ual theories have been oriented is the concrete-to- 
abstract dimension of learning. The writings of 
several other authors appear to substantiate this ba- 
sic concept presented by Finn, i.e., Gibson's (13) 
projective-conventional continuum, Osgood's (20) 
more detachable-less detachable continuum and 
Knowlton's (16) transparency-opacity continuum. 


Although it has been established that the use of 
carefully prepared visual aids can facilitate student 
learning, there has been little attempt to determine 
the relative effectiveness of the various types of 
visual illustrations in promoting student achieve- 
ment of varied educational objectives. Miller, et al. 
(18: 18-85) have stated that it would be a mistake to 
assume that one cue added to another would increase 
learning by a linear increment. Bruner, et al. (5) 
and Travers, etal. (24) have suggested that learn- 
ers do not need highly embellished stimuli in order 
to recognize the attributes of an object or situation 
or to identify the object or situation. The available 
evidence [ French (12), Miller (17), Rappaport (21), 
Travers (25), and Attneave (2)] seems to indicate 
that the effectiveness of discrimination learning pro- 
moted by the addition of relevant stimuli may be 
limited by the information processing capacity of the 
organism with learning reaching an early peak and 
then diminishing with the addition of relative, but su- 
perfluous, cues. Broadbent (3, 4) has explained that 
the reduction of learning as the cue stimulation in - 
creases is caused by the filtering process inthecen- 
tral nervous system which prevents much of the 
realistic stimuli from receiving active reception in 
the brain. 


SPECIFIC OBJECTIVES 


This study investigates the possible existence of 
a visual continuum representing learning effective- 
ness for eleventh-grade students. Specifically, this 
study attempts: 
a. to measure the relative effectiveness of eight 
different types of visual illustrations used to 
complement oral instruction; 
+ to determine at what point further increases 
of realistic detail in visual illustrations fail 
to produce Significant differences in the 
achievement of the five specific educational ob- 
jectives; 
€. to compare the a; 
layed retention г. 
and white and co 


H 
x PL ek to design and photograph nine 
y 2 inch slide Sequences for this Study 


ee 


Each sequence consisted of 39 slides. Five se- 
quences were in black and white and four were in color. 
The realistic photographic sequence (B and W) was 

the first slide sequence produced. From the realis- 
tic sequence a graphic artist reproduced the line 

representations and the detailed shaded drawings 

(both black and white and color versions). Extreme 

care was required to make certain that the same for- 
mat and size relationship that appeared in the orig- 

inal heart photographs were conveyed to the draw- 

ings and eventually to the slide sequences. 


The oral instruction was presented tothe students 
by means of a tape recorder. During the oral in- 
struction an audio signal cued the instructor to 
change the slide so that the presentation was syn- 
chronized. Thus, both the oral and visual instruc- 
tion presented the same information simultaneously. 


EXPERIMENTAL DESIGN 


The subjects were 264 eleventh-grade students 
drawn from the Farrell School System, Farrell, 
Pennsylvania, and the Sullivan County High School, 
Laporte, Pennsylvania. The number of students in- 
volved in the delayed retention was 252; this reduc- 
tion was caused by student absenteeism. 


The experiment was conducted so that classes of 
students were assigned at random to one of the nine 
treatment groups. Each student in each treatment 
group received a pretest, participated in his partic- 
ular instructional presentation, and then received 
four individual posttests. The students also partici- 
pated in a delayed retention testing session two 
weeks later at which time they received the same 
battery of tests; they had no prior awareness of the 
delayed testing session. 


The eight groups receiving visuals of the heart 
were considered to be the experimental groups and 
the group receiving slides on which appeared printed 
symbols of the terms being discussed was the con- 
trol group. Information transmitted to each student 
via the oral channel was held constant for each pre- 
sentation; the rate and number of words per minute 
were identical. The amount of realism possessed 
by the visual illustrations was varied for each of the 
eight experimental groups. Students viewed their 
respective slides for equal amounts of time. 


Analysis of covariance, using scores on the 36- 
item physiology pretest, was used to determine the 
significance of difference in immediate achievement 
for the nine treatment groups. An analysis of covar- 
iance was also used on the data obtained from the 
delayed retention testing with the immediate reten- 
tion total criterion test scores being used as the ad- 
justing variable. In testing for significance of dif- 
ferences among the means of the several treatment 
groups, Dunn's c- procedure [ Dunn (8); Sparks (23)] 


Was used to compare the means of the independent 
samples. 


CRITERION MEASURES 


Drawing Test 


The drawing test (N 


ч = 18 items) provided the stu- 
dent with a list of Spec 


ific terms corresponding to 
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the parts of the heart discussed in the oral pre- 
Sentation. The students were required to draw a 
Tepresentative diagram of the heart and place the 
numbers of the listed parts in their respective posi- 
tions. In this test emphasis was placed on the posi- 
tioning of the verbal symbols with respect to their 
Concrete referents, not the quality of the diagram 
they were to draw. (Kuder-Richardson Formula 21 
Reliability Coefficient: Immediate Retention . 88; 
Delayed Retention . 84.) 


Identification Test 
aCentilication Test 


This multiple choice test (N - 20 items) required 
students to identify the numbered parts ona detailed 
drawing of a heart. Each part of the heart, which 
had been discussed in the oral presentation, was 
numbered on the drawing, The Students were pro- 
vided with an answer sheet with corresponding num- 
bers on which they could select the name of the part 
of the heart which corresponded tothe numbe Y. 
(Kuder-Richardson Formula 21 Reliability Coetti- 
cient: Immediate Retention , 77; Delayed Retention 
474.) 


Terminology Test 


This test (N - 20 items) consisted of a series of 
fill-in questions; it was possible for one and only 
one specific answer to be Correct. This test at- 
tempted to measure the learner's knowledge of ref- 
erents for specific Symbols or knowledge of the ref- 
erent which would be most appropriately associated 
With the use of a Specific symbol. (Kuder- Richard- 
Son Formula 21 Reliability Coefficient: Immediate 
Retention . 69; Delayed Retention , 74.) 


Comprehension Test 


The comprehension test consisted of 20 multiple- 
choice items. Direct reference was made to the 
position of specific parts of the heart while function- 
ing. The student was then asked to identify the po- 
sition that other specified parts of the heart would 
be occupying at that particular moment. This test 
required that the students have a thorough under- 
Standing of the heart, its parts, its Internal function- 
ing, and the simultaneous processes which occur 
during the systolic and diastolic phases. The com- 
prehension test was designed to measure a ty pe of 
understanding that occurs when the individual under- 
stands what is being communicated and when he can 
use the information being received to explain some 
other phenomena. (Kuder-Richardson Formula 21 
Reliability Coefficient: Immediate Retention . 63; 
Delayed Retention , 60. ) 


Total Criterion Test 
—9ta! Criterion Test 


Scores received on the four individual criterion 
tests were combined in a composite 78- item crite- 
rion test designed to measure total understanding of 
the concepts presented. (Kuder- Richardson Formu- 
la 21 Reliability Coefficient: Immediate Retention 
: 90; Delayed Retention .91.) 


Physiology Pretest 


A physiology pretest consisting of 36 multi ple- 
Choice items was administered to all participants in 


C 
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an attempt to determine their prior factual kno w ]- 
edge of the functional aspects of human physiology. 

Scores on this test were used as the adjusting vari- 
able in the analysis of covariance technique to eval- 
uate the relative effectiveness of the various treat- 
ment groups receiving oral instruction complemen- 
ted by visual illustrations. (Kuder- Richardson Re- 
liability Coefficient: Immediate Retention . 61; De- 

layed Retention . 90.) 


TREATMENT GROUPS 
Oral Presentation: Group I 
sentation: Group I 


Students in Group I received oralinstruct ion 
throughout the entire presentation from a tape re- 


Spelling of the parts of the heart were projected on 
the screen as they were mentioned in the ora] pre- 
Sentation. There were no visual illustrations used 


Abstract Line Presentation (B and W): Group II 


Students in Group II received the same oral in- 
Struction as students in Group I. In addition, they 
received abstract line illustrations (black lines on 
a white field) depicting the form and the relative 
locations of the parts of the heart as they were men- 
tioned in the oral presentation. The line drawings 
used in this presentation were Similar to instr uc- 
tional drawings used in many current science text- 
books. 


Abstract Line Presentation (Color): Group III 


Students in Group III received the same oral in- 
Struction as students in Group I. In addition, they 
received abstract line visual illustrations (blue lines 
on à pink field) similar to those received by students 
in Group II; however, these illustrations we re in 
color. 


Detailed, Shaded Drawing Presentation ( 


Group IV 


Students in Group IV received the same 
Struction as students in Group I. 
viewed detailed, Shaded draw 
Parts of the heart as they were mentioned in the oral 
instruction, These drawings were more complex 


ings and more realisti- 


B and W): 


Students in Group V received the sa me oral in- 
struction as Students in Group I. 


Heart Model Presentation (B and №): G 


Group vi 


he Same ora] in- 
In addition. they 
odel as the heart 

iscusseq orally, 


Students in Group VI received t 
Struction as students in Group I. 
viewed photographs of a heart m 
and its related parts were being d 
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Heart Model Presentation (Color): Group VII 


Students in Group VII received the same oral in- 
struction as students in Group І. In addition, they 
viewed colored photographs of a heart model as the 
heart and its related parts were being discussed 
orally. 


Realistic Photographic Presentation (B and W): 
Group VIII 


Students in Group VIII received the same oral in- 
Struction as students in Group I. In addition, they 
viewed realistic black and white photographs of the 
heart as the heart and its related parts were being 
described orally. 


Realistic Photographic Presentation (Color): 
Group IX 


Students in Group IX received the same oral in- 
Struction as students in Group І. In addition, they 
viewed realistic colored photographs of the heart 
Similar to the black and white Sequence received by 
Students in Group VIII. 


SUMMARY OF ANALYSES: IMMEDIATE 
RETENTION 


On the total criterion test and on three of the 
four individual criterion tests, significant differ- 
ences were found to exist among the means of the 
nine groups. The F ratios for the adjusted analy- 
ses were all significant beyond the 0.01 level with 
the exception of the terminology test where no sig- 
nificant differences were found to exist. 


A. On the total criterion test the various slide 
Sequences facilitated total student learning 
with differing degrees of effectiveness. 


1. The abstract line presentations (B and W 
and Color), the detailed, shaded drawing 
presentation (Color), and the realistic 
photographic presentation (B and W) were 


more effective than the oral presentation 
alone, 


The oral presentation alone was as effec- 
tive as the detailed, sh 


presentation (Color). 


3. When the black and whi 
of each visual presenta; 
in terms of their instr. 
ness, no significant d 


te and color versions 
tion were compared 

uctional effective- 
ifferences were found. 


B. In relation to the i i 
promotion of student ach E 
ment on the drawing test: — 


1. The abstract line 
and Color) and the detai 
ing presentation (Color 
tive than the oral pres 


2. The oral presentation alone 
tive as the detailed, Shaded 
Sentation (B and W), the he: 


Was as effec- 
drawing pr e- 
art model pre- 
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sentations (B and W and Color)and the 
realistic photographic presentations (B and 
W and Color). 


3. When the black and white andcolor versions 
of each visual presentation were compared 
in terms of their instructional effectiveness, 
no significant differences were found. 


C. In relation to the promotion of student achieve- 
ment on the identification test: 


1. The abstract line presentation (B and W), 
the detailed, shaded drawing presentation 
(Color), and the realistic photographic pre- 
sentation (B and W) were more effective 
than the oral presentation alone. 


2. The oral presentation alone was as effec- 
tive as the abstract line presentation 
(Color), the detailed, shaded drawing pre- 
sentation (B and W), the heart model pr e- 
sentations (B and W and Color), and the 
realistic photographic presentation (Color). 


3. The detailed, shaded drawing presentation 
(Color) was more effective than the detailed, 
shaded drawing presentation (B and W), and 
the heart model presentations (Band W and 
Color). 


D. In relation to the promotion of student achieve- 
ment on the terminology test, The F-ratio did 
not exceed the critical value for а 0. 05 level 
test. Further analysis among the various 
group means was not justified, 


E. In relation to the promotion of student achieve- 
ment on the comprehension test: 


1. The oral presentation alone was as effec- 
tive as each of the visual presentations. 


2. The abstract line presentations (Band W 
and Color), the detailed, shaded drawing 
presentations (B and W and Color), and 
the realistic photographic presentation (B 
and W) were more effective than the realis- 
tic photographic presentation (Color). 


INTERPRETATION OF ANALYSES: IMMEDIATE 
RETENTION 


In terms of instructional effectiveness, economy, 
and simplicity of production: 


l. The abstract line presentation (Band W) 
Should be used to promote total student under- 


Standing of the concepts presented inthe in- 
Struction. 


The abstract line presentation (B and W) 
Should be used to promote student learning of 
Specific locations of the various patterns, 


Structures, and positions ofthe parts of the 
heart, 


3. Theabstr a 
Should be us 
learning, d. 


Ct line presentation (B and W) 
ed to promote student transfer of 
e. , the ability to identify the 


> 
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numbered parts on a diagram of the heart 
from information presented via the oral-vis- 
ual presentation, 


- The oral presentation alone should be used to 


promote student learning of the refere nts 
of terms. 


- The oral presentation alone should be used to 


promote the development of a new view, or 
reorganization, of the material by the student, 


SUMMARY OF ANALYSES: DELAYED RETENTION 


In relation to the promotion of student achieve- 
ment on the total criterion test, the drawing test, 
the identification test and the comprehension test, 
the F-ratio did not exceed the Critical value for a 
0. 05 level test. Further analysis among the various 
group means was not justified, Significant differ- 
ences were found to exist among the means on the 
terminology test (0. 05 level). 


A. In relation to the promotion of student achieve- 


ment on the terminology test: 


1. The oral presentation alone was as effec- 
tive as each of the visual presentations, 


2. When the black and White color versions of 
each visual presentation were compared in 
terms of their instructional effectiveness, 
no significant differences were found. 


INTERPRETATION OF ANALYSES: DELAYED 
RETENTION 


In terms of instructional effectiveness, economy, 


CONCLUSION 


Emerging from the Study are a number of gener- 
alizations for the use of visual illustrations for pro- 
moting the achievement of Specific learning objec- 
tives in the biological sciences by eleventh-grade 
Students. Further research is needed, however, to 
determine whether the same types of visual illustra- 
tions are equally effective in Promoting student 
learnings of subject matter in different content 
areas and for students in different grade levels, 


1. The visual realism Continuum is nota reliable 


predictor of learning efficiency for eleventh- 
grade students. The findings of this study 
Seem to substantiate the hypothesis that real- 
ity may be edited for instructional purposes. 


This would seem to raise a number of impor- 6. 


tant questions regarding the instructional val- 
ue of many types of visual aids being produced 
commercially. 


Sured in this study by the total criterion test, 


the drawing test, and the identification test. 


5 


3. In terms of instructional effectiveness, econ- 
omy, and simplicity of production the abstract 
line representations (B and W) should be used 
to complement eleventh-grade instruction in 
this study by the total Criterion test, the draw- 
ing test and the identification test, 


4. Color in visual illustrations Presented to 
eleventh-grade Students does not appear to be 
an important variable in increasing student 
learning of educational Objectives measured 
in this study. 


5. Although it was found that the presentations 
using visuals were just as effectiveasthe oral 


ment on the delayed retention testing, extreme 
caution is necessary in the interpretation of 
this finding. Since the retention tests were 
administered two weeks after the immediate 


FOOTNOTE 


1. This article is based on research sponsored by 
the U.S. Department of Health, Education, 


described can be found in the Final Report 
Project No. 6-8840, Grant No, OEG-1-7- 
068840-0290 by F. M. Dwyer, Jr. » Which in- 
cludes this and other Studies, 
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DELIBERATIVE ATTITUDES, IQ, AND 
SOCIAL CLASS 


T. BENTLEY EDWARDS 
University of California, Berkeley 


ABSTRACT 


When large numbers of children are polled concerning their attitudes toward school and academic st udy, 
those from poor homes are, on the average, less likely to hold favorable attitudesthan those from better homes. 
This fact is hardly surprising. But the relationship between favorable attitudes and socio-economic status is 
far less marked than is the relationship between IQ and socio-economic status. Hence, whenchildren are 
selected for high-ability classes and eventually for college onthe basis of IQ or related measures, a child who 
likes ideas is far more likely to be rejected by the school system if he comes from the lower socio-economic 
classes. The foregoing conclusion is based on a survey of thousands of children in elementary and high schools 


in this country and in England. 


ATTITUDES FAVORABLE TO DELIBERATION 


Elsewhere, the development of scales to meas- 
ure attitudes favorable to the acquisition of under- 
standing needed for deliberation prior to the making 
of choices has been described (2). Attitudes favor- 
able to the study of things as opposed to people have 
been separated and labeled Theoretic as opposed to 
Prudent. The fate of children with Theoretic inter- 
ests has been discussed (3). Although possession of 
either Theoretic or Prudent interests was shown to 
be advantageous for school achievement, children 
with Prudent attitudes received disproportionately 
more of school rewards, that is high grades, and 
those with Prudent as opposed to Theoretic interests 
were found more likely to be selected for advanced 
educational opportunities. 


In the present article, data will be presented to 
show that although scores on the so-called Intelli- 
gence Tests correlate highly with socio-economic 
status, a fact which has been established by data 
from many investigations (5, 4, 1, 6), scores on 
attitude tests do not. There is a correlation, but it 
is much less pronounced. Hence, whenchildren 
and youth are selected for advanced educationon the 
basis of intelligence test scores, or on other test 
Scores or marks that correlate highly with intelli- 
gence test scores, many children with strong delib- 
erative interest scores are rejected. These days, 
when concern for equality of educational opportunity 
is so urgently voiced, fresh tests are needed with 
which to distinguish those members of the less fav- 
ored social classes, many of whom are Negroes, 
who might respond to a more enlightening school en- 
vironment than the one they now encounter. 


SOURCES OF DATA 


Attitude tests were administered to 3, 750 students 
in 12 high schools around the San Francisco Bay. In- 


telligence test scores, and data on occu pation of 
father were collected for the same students, 


The 12 schools were chosen so as to provide as 
wide a range of intelligence and occupation of father 
as possible rather than to provide a representative 
sample. To this end, census data were analyzed 
and schools selected that seemed likely to produce 
generous samples of rural and urban residents; up- 
per, middle, and lower social classes; inhabitants 
of comfortable suburbs, of industrial slums, and of 
pockets or urban wealth; and of Negro and of white 
members of the various socio-economic classes, 
The assumption taken here is that when Such socio- 
economic variations are sampled, the full range of 


attitude and of intelligence test scores is likely to 
be included. 


To facilitate data collection, schools within driv- 
ing distance of Berkeley were used, and only those 
Schools eager to cooperate in the st udy were in- 
cluded. Descriptions were written for each of the 
60 odd high schools within a 50- mile radius of the 
University. These schools were divided into four 
major groups according as they were found in areas 
classed by the Bureau of the Census as: metropoli- 
tan, urbanized areas, cities over 5,000 and places 
under 5,000. In the first of these, three lar 
metropolitan schools were selected mor is 
arbitrarily to include an all N. 
No. 11, Table 1), School No. 
from several social classes, 
used in a pilot study. 
2) in the metropolitan 


academic courses. 


Attempts were made to achieve а represent ti 
ative 
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TABLE 1 


DISTRIBUTION OF FATHERS’ OCCUPATIONS 
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Small 
Manual White Business Farm  Profes- Execu- 

Number Community Setting Labor Collar and Mer. Owner  sional tive 
1 Town--rural marketing center 42% 20% 19% 6% 1% 1% 

2 Metropolitan--boy's private school 6 24 18 0 30 21 

3 Metropolitan-- heterogeneous 38 33 16 1 8 4 

4 City--light industry 52 23 6 4 12 3 

5 Town--rural 43 20 12 19 5 1 

6 Town--rural 33 32 17 8 7 3 

7 City--industrial 47 29 17 0 6 1 

8 Urban--working class residential 55 32 11 0 1 1 

9 Suburban 23 36 17 0 16 8 

10 Urban--industrial 67 18 12 0 3 0 

11 Metropolitan-- Negro residential 74 14 6 0 6 0 

12 Urban--upper class residential 2 19 19 0 25 35 

13 Metropolitan-- **white collar” residential 29 33 25 1 12 0 

All Schools Combined 40% 27% 16% 39 9% 5% 
TABLE 2 

DISTRIBUTION OF FATHERS’ EDUCATION 

Fathers’ education School Number 

terminated during or All Schools 
at conclusion of: 1 2 3 4 5 6 T 8 9 10 11 12 13 Combined 
Elementary school 25% 3% 19% 28% 46% 17% 37% 28% 11% 23% 32% 0% 11% 22% 
High School 51 18 49 48 35 з 49 57 42 60 40 20 45 45 
College 19 44 21 13 13 17 9 9 34 8 16 50 31 21 
Post-graduate 5 34 9 8 4 1 4 $ R 0 4 26 10 9 
Not available 0 1 @ $9 9? атха 1 9 8 4 3 3 


Sample in the Negro high school and the heterogene- 
ous School. In the pilot study school, al most the 
entire population of Grade 11 was tested, aswasthe 
entire population from the private school. Thetotal 
number of students selected from each ar ea was 
Chosen so as to match as nearly as possible the pro- 


portions of the population of th i i 
living in such a ehe of the State of California 


Tables 2, 3 — à 
tèns’ варено: ч: 4 show the distributions of fa - 


their racial origin 


the profes- 
Classes. For school No 2 si 
end for School No. 12 two perce nt of the 

ork with their hands, In each of these two 


schools one father in five operates a small business. 


None are farmers. In school No. 2, 31 percent, 
and in school No. 12, 39 percent of the children 
show IQ's in excess of 120. In school No. 2 only 
one child in a hundred shows an IQ of less than 100, 
In school No. 12, 13 percent show an IQ of less 
than 100. School No. 2 is 97 percent white; school 
No. 12 is 100 percent white. 


By way of contrast, 46 percent and 37 percent of 
the fathers in schools No. 5 and No. 7 did not attend 
high school. Between 40 percent and 50 percent of 
these fathers work with their hands, Nearly a third 
of the children in these schools show an IQ of 100- 
110. In school No. 5, 96 percent, and in school No. 
7, 86 percent of the children are white. The first 
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TABLE 3 


DISTRIBUTION OF IQ TEST SCORES 


School Number 


All Schools 


IQ Scores 1 2 3 4 5 6 7 8 9 10 11 12 13 Combined 
150 plus оо 0% o% 0% 0% O% 0% 0% 0% 0% ow 1% 1% 0% 
140-149 0 1 1 0 0 1 1 0 0 1 0 2 2 1 
130-139 1 6 3 2 1 3 1 1 2 0 0 10 6 2 
120-129 6 24 12 3 3 4 7 4 7 9 1 26 11 8 
110-119 21 43 20 15 17 15 23 12 16 9 8 27 24 18 
100-109 271 25 22 9 38 231 з 16 22 28 22 16 2 20 
90-99 23 1 15 10 21 15 21 14 16 23 24 11 12 15 
80-89 12 0 14 3 5 12 6 1 5 235 25 2 т 9 
70-79 5 0 Т 2 2 1 1 2 2 1 дт 0 2 4 
60- 69 0 0 1 0 0 1 1 1 0 0 1 0 0 1 
Below 60 0 0 1 0 0 0 0 1 0 0 0 0 0 0 
Not Available 5 0 4 56 13 27 8 42 30 6 2 5 14 22 
Test Mix Otis T-M CMM CMM-, CMM Otis T-M Otis CMM T-M T-M T-M 
TABLE 4 


RACIAL DISTRIBUTION 


All Schools 
Race 1 2 3 4 5 6 7 8 9 10 11 12 13 Combined 
Negro 0% 1% 29% 1% 0% 1% 10% 1% 0% 22% 91% o% 1% 13% 
Oriental 0 2 10 0 4 2 3 1 0 4 2 0 6 3 
White 100 97 60 97 96 96 86 97 99 71 5 99 86 83 
Not Available 0 0 1 2 0 1 1 1 1 3 2 1 1 1 
Number of Cases 388 160 380 227 135 141 222 538 288 123 311 96 601 3750 


of these two schools is in a small town in the coun- 
try; the second is in a small heavily industrialized 
town. 


Schools No. 8 and No. 10 are in heavily industri- 
alized communities within the metropolitan district, 
Fathers of children in No. 8 are nearly all in the 
working classes; 55 percent work with their hands; 
32 percent are white collar workers, and 11 percent 
own small businesses. Fathers of children in No. 
10show 67 percent, 18 percent, and 12 percent in 
these three categories. Although 57 percent and 60 
percent of these fathers graduated from high school, 
only 12 percent and 8 percent are reported as col- 
lege graduates, Only 60 percent of the IQ’s from 
school No. 8 were available, but these showa fairly 
normal distribution, Those from school No. 10, 
however, show 23 percent from 90-99 and 23 per- 


cent from 80-89, 
the children are white, 
cent are white, 


In one other School, No, 
children are Negroes, 


graduated from high schi 


In school N 


о. 8, 97 percent of 


In school No, 10, 71 per- 


11, 91 Percent of the 


acted fro 
П. None of these m 
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INTELLIGENCE TEST SCORES 


All schools do not give the same tests of intelli- 
gence. Inthe 12 schools selected, the Terman- Mc- 
Nemar was given most frequently, but some of the 
schools gave the Otis, and others gave the Califor- 
nia Test of Mental Maturity. 


Correction factors are available, or could be 
worked out to make scores on these various test s 
closely comparable, but because only gross com- 
parisons were to be made in the present study, cor- 
rection factors have not been applied. The neces- 
sary information is available and the increased ac- 
curacy may be sought at a later date. 


So far as the present study is concerned, the cor- 
relations indicated would probably be increased 
slightly if the data on IQ were more accurate. No 
change in the conclusions drawn could be anticipated. 


FATHERS' OCCUPATION 


The students themselves were asked to report 
their fathers' occupation. Attached to the attitude 
inventory was a personal data sheet which included 
the following item: 


22. My father's (or step-father's) job or 
occupation is 
Just what does your father do in his 
job? 


The explanation called for in the second part of the 
question helped to decide whether the child really 
knew what his father did at his work. Further cor- 
roboration was supplied when the child was asked 
about his father's education. Unless the various re- 
sponses were in agreement, the datum on occu pation 
was not used. An elaborate key was prepared with 
which to classify the father's occupational status. 
This key includes the orientation of the occupation 
according to the four poles of the attitude inventory, 
but the orientation of the occupation does not enter 
the argument of the present article which refersonly 
to the socio-economic status of the father's occupa- 
tion. The complete key is provided in Table 5. 


ATTITUDES 


The attitude inventory has been published (2), but 
for convenience several Sample items from the Pru- 
dent-Immediate, Theoretic- Immediate, and Prudent- 
Theoretic scales are reproduced in Table 6, 


Both Prudent and Theoretic orientations are as- 


Sumed to indicate a likin i i 
r g for mental deliber 
prior to choice- makin, nm 


matics, and 


Е engineering. 
vide the nee б 


ded contrast a 


for more immediate and proximate clues in the so- 
lution of life's daily problems. Conversation, per- 
sonal experience, and intuitive feeling are more 
likely to govern the choices made than are the use 
of principles and modes of thought learned in study- 
ing formal school subjects or disciplines. 


ASSOCIATIONS INDICATED 


Tables 9 - 12 show the association of Prudent or 
Theoretic as opposed to Immediate orientations with 
IQ. Tables 7 and 8 show Prudent as opposed to 
Theoretic orientations and IQ. These are offered 
by way of contrast. Scores are tabled separately 
for boys and girls, and separation into three groups 
according to fathers' occupation is also provided. 


Tables 9 - 12 show that although IQ i s linked 
closely with occupation of father, a deliberative at- 
titude, whether Prudent or Theoretic, is less close- 
ly linked with either IQ, or occupation of father. 
This means that when children are selected for ad- 
vanced educational opportunities by means of IQ 
Scores, the children of working class parents are 
at a disadvantage whether they have deliberative at- 
titudes or not. This is also true when children are 
selected for high ability groupings by means of IQ 
Scores or by other test scores that correlate highly 
with IQ. The foregoing is based on Prudent-Imme- 
diate and Theoretic-Immediate scores. Prudent- 
Aesthetic and Theoretic-Aesthetic scores lead to 
the same conclusions. 


Attitude scores can vary only from one through 
four. It is important to remember that by the scor- 
ing technique, approximately one quarter or 25 per- 
cent of the population studied was arbitrarily as- 
signed each of the four ranks. The percentage of 
Prudent or Theoretic in each of the various classi- 
fications should normally be 45-55, and departure 
from this norm must be offset by a similar depar- 
ture in another grouping. For example, 44 percent 
of the boys are Prudent as opposed to Theoretic. 
But 76 percent of the girls are Prudent. Among 200 
students, half boys and half girls, we should there- 
fore find 120 (76 plus 44) who are Prudent as op- 
posed to Theoretic. The reason for the large num- 
ber of Prudent as opposed to Theoretic is that the 
cut-off points were decided on the basis of responses 
from a pilot group which was more Theoretic than 
the more general sample. The resulting imbalance 
in no way interferes with the argument of the pres- 
ent article. 


By contrast, 47 percent of the boys and 58 pe r- 
cent of the girls are Prudent as opposed to Immedi- 
ate. Hence among 200 students, we should find 105 
(47 and 58) who are Prudent. 


The number of 100 in any of the cells, unless the 
population of the cell were small, would mean heavy 
depletion in some other cell. For example, 91 per- 
cent of the high IQ daughters of professionals are 
Prudent on the Р-1 Scale. This is offset by 30 per- 
cent for the low IQ boys. Thetotal percentage of 
Prudent for the girls is 58. This is balanced by a 
total of only 47 for the boys. For the Bay Area 
Study as a whole, 1,315 out of 2,518, or 52 percent 
Show Prudent Scores, when allthose scoring one 
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TABLE 6 


SAMPLE ITEMS 
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Prudent-Immediate Scale 


Theoretic-Im mediate Scale 


Prudent-Immediate Scale 


Science has definitely not béen 
able to show that colored races 
are inferior to white races. 


A business man should make his 
decisions strictly according to 
the interests of his business. He 
should not worry about what hap- 
pens nationally to wages and 
prices. 


I should rather be elected to the 
Student Council than be selected as 
an honor student in science. 


I would rather be known as the 
writer of a social column pub- 
lished in many papers than as 
the Director of an astronomical 
observatory. 


Iam more interested in finding 
out how TV has affected people’s 
taste than in finding out how TV 
works. 


Iam more interested in following 
newspaper reports on the recent 
discoveries regarding ‘‘negative 
matter" than on the develo p- 


ments in racial integration in the 
Schools. 


TABLE 7 


BAY AREA--BOYS P - T (PERCENTAGES PRUDENT BY IQ AND FATHER'S OCCUPATION) 


IQ 
Occupation of Father 120« 110-119 100-109 90-99 -89 Total 
Professional and executive 49 39 45 27 37 43 
84 88 58 11 8 249 
Percentage of class in each IQ bracket 34 36 23 4 3 100 
White collar 43 50 36 40 41 43 
N 82 137 129 82 41 471 
Percentage of class in each IQ bracket 17 29 28 18 8 100 
Manual worker 45 40 40 40 62 45 
N 58 120 174 141 138 631 
Percentage of class in each IQ bracket 9 19 28 22 22 100 
45 44 40 39 
Totals 56 
N 224 345 361 234 187 ee 
Percentage in each IQ bracket 17 25 27 17 14 100 


and those scoring two on the P-I scale are added to- 
gether. This means that 100 - 52, 48 percent of the 
Same group of students are classified as Immediate. 


We are now ready for a more detailed study of 


Tables 7 - 12. 


PRUDENT- THEORETIC 


than a third, 34 


Attention to percentage of class in each IQ bracket 


shows the association of IQ with social class. More 


is group showing a 
44 percent 
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TABLE 8 


BAY AREA--GIRLS P - T (PERCENTAGES PRUDENT BY IQ AND FATHER'S OCCUPATION) 


IQ 
Occupation of Father 120+ 110-119 100-109 90- 99 -89 Total 
Professional and executive 82 16 65 94 33 78 
N 45 42 26 17 3 133 
Percentage of class in each IQ bracket 34 32 20 13 2 100 
White collar 84 78 15 68 74 76 
N 74 111 133 15 50 443 
Percentage of class in each IQ bracket 16 25 30 17 11 100 
Manual worker 80 87 75 68 78 16 
N 59 121 179 155 129 643 
Percentage of class in each IQ bracket 9 19 28 24 20 100 
Totals 82 80 74 10 76 76 
N 118 274 338 247 182 1219 
Percentage in each IQ bracket 14 23 28 20 15 100 
TABLE 9 
BAY AREA--BOYS P - I (PERCENTAGES PRUDENT BY IQ AND FATHER'S OCCUPATION) 
IQ 
Occupation of Father 120+ 110-119 100-109 90-99 -89 Total 
Professi i 
E dud and executive 61 59 53 36 16 56 
84 88 58 11 6 241 
Percentage of class in each I : 
n each IQ bracket 34 36 23 5 2 100 
Whit 
ite collar 52 50 43 
N 82 41 30 45 
: 137 129 82 33 463 
ercentage of class i 
in each IQ bracket 18 29 28 18 1 100 
Manual worker 
N He 49 45 38 31 44 
" 8 120 174 141 96 589 
ercentage of cl: i 
g class in each IQ bracket 10 21 29 24 16 100 
Totals 
N 29 52 46 39 3 
224 345 361 234 А ra 
р А 135 1299 
ercentage in each IQ bracket 
i 27 28 18 
10 100 
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TABLE 10 


BAY AREA--GIRLS P - I (PERCENTAGES PRUDENT BY IQ AND FATHER'S OCCUPATION) 


IQ 
Occupation of Father 120. 110-119 100-109 90-99 -89 Total 
rofessional and executive 91 69 58 59 67 73 
uh ^ 45 42 26 17 3 153 
Percentage of class in each IQ bracket 34 52 20 13 2 100 
е White collar 18 80 60 55 48 60 
N 74 111 133 75 50 443 
Percentage of class in each IQ bracket 16 25 30 17 11 100 
Manual worker 64 64 56 48 40 53 
N 59 121 179 155 129 643 
Percentage of class in each IQ bracket 9 19 28 24 20 100 
Totals FI 65 55 51 42 58 
N 178 274 338 247 182 1219 
Percentage in cach IQ bracket 14 23 28 20 15 100 
^ 
TABLE 11 
BAY AREA--BOYS T - I (PERCENTAGES THEORETIC BY IQ AND FATHER'S OCCUPATION) 
IQ 
Occupation of Father 120+ 110-119 100-109 90-99 -89 Totals 
Professional or executive 64 82 71 64 50 
N 84 88 58 11 i 
8 249 
Percentage of class in each IQ bracket 36 36 23 5 2 xéti 
^ White collar 63 63 63 56 
59 
N 82 137 2 62 
d 82 41 471 
Percentage of class in each IQ bracket 18 29 28 18 s: 
100 
Manual worker 71 63 59 
N 58 120 174 Mi a 62 
136 629 
Percentage of class in each IQ bracket 10 21 29 94 
š 16 100 
Totals 6 68 63 
59 
N 224 345 361 234 60 64 
185 1349 
Percentage in each IQ bracket 17 27 28 is 
10 100 


THE JOURNAL OF EXPERIMENTAL EDUCATION 
16 


TABLE 12 


, T 
BAY AREA--GIRLS T - I (PERCENTAGES THEORETIC BY IQ AND FATHER'S OCCUPATION) 


IQ 
Occupation of Father 120- 110-119 100-109 90-99 -89 Total 
1 50 
Faotéssicuad and executive 19 А ae їр E: 133 
Percentage of class in each bracket 34 32 20 13 2 pi 
wg та 4 9 а 
Percentage of class in each bracket 16 25 30 17 11 p 
жура ай л а а а d 
Percentage of class in each bracket 9 19 28 24 20 100 
— 118 4 338 мл 182 1219 
Percentage in each IQ bracket 14 23 28 20 15 100 
bracket for each social class shows that boys gener- The point to be made is that orientation of attitude | 
ally are more likely to be Theoretic than Prudent. 


The percentage varies around a mean of 44. Among 
the upper class boys, half the high IQ's are Prudent, 
but of the low IQ's only about a third are Prudent. 


For this group of boys the 
spond to the items may be 
her groups; a substantial 


г In turn, they та; 
Science and technology to re 


ve been deceived about th 
and history by their experi 


y know enough about 
pudiate them, and to 
€ concerns of literature 
ences in low ability classes. 
Three of ever. Tour gi E 
gestion varie d girls are Prudent and the Pro. 
with IQ. 


is much less firmly linked with social class than is 
IQ. So far as the Prudent- Theoretic contrast is con- 
cerned, no severe loss of deliberative orientation 
can be attributed to selection on the ba Sis of IQ or 
on the basis of tests that correlate with IQ, since 
Prudent and Theoretic orientations are both deliber- 
ative. But when attention is paid to scales contrast- 
ing deliberative and non-deliberative attitudes, sim- 
ilar absence of correlation with social class pro- 
duces an unexpected and disturbing result when 
children are selected on the basis of IQ. 


PRUDENT-IMMEDIATE 


For boys, the association between Prudent a nd 
high IQ is about the same as it is for girls but for 
neither sex is the association as marked as isthe y 
link between high IQ and high socio-economic status. 
For example, 61 percent of the sons of members of 
the professional and executive class withanIQ above 
120 have Prudent as opposed to Immediate ta stes, 
whereas for sonsof professionals with an IQ between 


90 and 99 (chosen because the number below 90 was 


too small) the proportion is 36 percent. For girls 
the figures are 91 percent and 59 percent. But the 
Proportion of boys with an IQ in excess of 120 is 
three and a half, 34 to 10, when the percentages 
among the professionals and manual workers is com- 


pared. For girls, the proportions are almost the 
same, 


This incongruity is produced by the fact that the 
k between social cla 


ss and a Prudent orientation 


linl 
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is light. Slightly more than half, 56 percent, of the 
Sons of professionals are Prudent as opposed to Im- 
mediate. But 44 percent of the sonsof manual work- 
ers are also Prudent. For girls, the link between 
social class and a Prudent orientation is greater 
than for boys but still not nearly as markedas is the 
link between social class and high IQ. Nearly three- 
quarters, or 73 percent of the daughters of profes- 
sionals are Prudent; just over half, or 53 percent of 
the daughters of manual workers are Prudent. 


THEORETIC-IMMEDIATE 


The Theoretic-Immediate comparisons with IQ 
and social class show somewhat similar relation- 
ships except that the differences between boys and 
girls are reversed and the association of a Theo- 
retic preference with IQ is less marked. In fact, a 
Theoretic as opposed to Immediate preferenceisas 
likely to be found among the low IQ girls as among 
the high IQ girls and almost as likely among the low 
as among the high IQ boys. More than 60 percent 
of boys of all social classes show a Theoretic pref- 
ercence. Among girls, this preference is ex- 
pressed by between 40 and 50 percent. 


SELECTION BY IQ 


Tables 13 and 14 show the results of the associ- 
ations that have been described in a more striking 
fashion. For both boys and girls the two columns 
are nearly identical. Only one pair of columns will 
be described, the other three pairs of columns pro- 
viding strong support for the argument. Consider 
data for boys in the first two columns of Table 13. 
When boys are selected for School rewards leading 
to further educational opportunities upon a basis of 
IQ scores, 66 percent of the sons of the profession- 
alclass were so selected, but only 28 percent of the 
sons of manual workers, Because the sons of man- 
ual workers are as likely to indicate a preference 
for Prudent as opposed to Immediate activities as 
are the sons of professionals and executives, 72 per- 
cent of the Prudent among the sons of the profes- 
sionals will be selected but only 34 percent of the 
Prudent among the sons of manual workers, 


Table 15 uses a cut-off point of 120, a more re- 
alistic figure where the educational opportunities 


leading to the professions is concerned. Now, half 
the deliberative sons of professionals are being 
turned down, but four-fifths, or 80 percent, of the 
sons of manual workers with deliberative orienta- 
tions are being refused. 


Where selections have been made either on the 
basis of IQ or on the basis of measures that corre- 
late highly with IQ a number of potential failuresare 
known to have been admitted. The alleged stability 
of the IQ derives largely from its manufactured high 
relationship with past achievement. The IQ will pre- 
dict accurately only for those children who continue 
as they have in the past. Selection of children below 
the arbitrary IQ cut-off point who have strong delib- 
erative orientations, especially among the culturally 
deprived groups, might turn out to be a successful 
practice, as well as being more desirable on humane 
and compassionate grounds. 


TABLE 13 


PERCENTAGES SELECTED FROM TOTAL NUM- 
BER OF PRUDENT (AS OPPOSED TO IMMEDIATE) 
CHILDREN IN EACH SOCIAL CLASS, GIVEN A 
CUT-OFF POINT OF IQ = 110 


Boys Girls 
Father’s 1Q over IQ over 
Occupation 110 Prudent 110 Prudent 
Professional and 
executive 66 72 70 "4 
White collar 41 49 47 5з 
Manual worker 28 34 30 37 


TABLE 14 


PERCENTAGES SELECTED FROM TOTAL NUM- 
BER OF THEORETIC (AS OPPOSED TO IMMEDI- 
ATE) CHILDREN IN EACH SOCIAL CLASS, GIVEN 
A CUT-OFF POINT OF IQ - 110 


Boys Girls 
Father's IQ over IQ over 
Occupation 110 Theoretic 110 Theoretic 


ums RN 


Professional and 


executive 70 71 66 68 
White collar 47 48 41 39 
Manual worker 30 30 28 28 
TABLE 15 


PERCENTAGES SE LECTED 


Prudent p - 1 Th " 
Father's 19 over Extreme o tie T - 1 
Occupation 120 Extreme IQ over Extreme 


-score 120 т 


~score 


executive 34 E 
White collar 18 а 1; A 
Manual worker 10 19 i 
10 
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ABSTRACT 


The Dogmatism Scale for adults was adapted to provide an instrument for use in stud 
tion of belief-disbelief systems of children in grades four-six, 
of the forced-choice type. It can be administered to groups. 


termine a respondent's placement along an openminded- 


scorers at P< . 05. 


beliefs will lead to a better understanding of human 
behavior than currently exists and to implications 
for persons involved in the teaching process (5). 
He suggested that anindividual possibly possesses 


disbeliefs into Systems having descr ibable and 
measurable properties and observable behavioral 
consequences, and that such systems may be meas- 
ured along an openminded-closedminded continuum 
through a sampling of that person's reactions to a 


Thus, a person having a strong belief to the ef- 
fect that “the United States and Russia havejust 
about nothing in common" may tend to accentuate 
differences and minimize similarities, an indication 
that the individual may be more closedminded than 
openminded (5:61-62). Rokeach postulated a number 
of such properties of belief-disbelief Systems. He 
then wrote and collected statements designed to tap 
cognative aspects of such properties (5:73-80). 


The statements were compiled into a fo reed- 
choice, summated-rating type of scale designed to 
measure an individual's placement along the open- 
closed continuum. A respondent assigned one of six 
Possible responses, ranging from “strongly agree” 
to ‘‘strongly disagree, "' to eachitem. Differing re- 
Sponses were assigned differing numerical weights 
for scoring purposes. 


Rokeach and other investigators used scale 


scores as the criterion to form groups of individu- 
als presumed tobe either high or low along the 


(0. o 7—7 


closedminded continuum. 
pupils in grades four through six in all five elementary schools in a central Indiana 
tion, the instrument's reliability coefficient was found to be 


ies on the organiza- 
The paper-and-pencil scale contains 50 items 
A summated-rating scoring system is used to de- 
When administered to 485 
city of about 7,000 popula- 
.67. All items discriminated between low and high 


of such extreme groupson tasks other than respond- 
ing to items but also posited to follow from theoret- 
ical postulations were compared (5:99-410). To the 
extent that a group's performance of such tasks 


In a series of investigations it was found, 


for ex- 
ample, that groups measured as openminded by the 
Scale, incomparison to Broups measured as closed- 


minded, usually were better able to synthesize а 
series of beliefs needed to solve problems based on 
a fictitious miniature cosmology ( 5:171- 284 i 
Sons having such ability, Rokeach reasoned, would 
have belief-disbelief subsystems possessin 
degrees of intercommunication 
ty of belief-disbelief Systems, than would 


e 
not having such an ability Persona 


variety of Situations. Most of the re 
on Rokeach's postulations have invi 
dents and other adults, 
and conducted a 


SCALE ADA PTATION 


The development of a form 
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tion of reducing vocabulary and concept loads. The 
rewrittenitems, along withoriginalitems, were sub- 
mitted to juries of reading and psychology experts 
for evaluations of parallelism and to establish con- 
tent validity. The reading level of rewritten items, 
as determined through use of the Spache formula (6) 
and the Dale word list (2), was found to be at about 
grade 3. 0. 


The response schema for adult scale items, in- 
volving selection of one of six degrees of agreement, 
was changed to a system involving three degrees: 
agree (+), uncertain (0), or disagree (-). In scoring, 
a numerical weight of one was assigned to disagree 
responses, twoto uncertain responses, and three to 
agree responses. Sinceall items were written so 
that agreement was a presumption of closedminded- 
ness, the higher a respondent's score, the more 
closedminded he was presumed to be, 


Rewritten items were given trials with pupils en- 
rolled in grades four through six at schools in cen- 
tral Indiana towns. Sample populations were des- 
Cribed as predominantly white, middle class, Pro- 
testant, native born and of northern Euro pean 
descent. Children were giventhe impression that 
their responses to scale items would be anonymous. 
On a title page, respondents were asked to supply in- 
formation on their sex, age, and date of birth, but 
not their names. Data from school records were 
matched with that supplied by respondents to identify 
individuals. Teachers identified pupils whom they con- 
sideredto be incapable of reading the items and data 
on such pupils were eliminated from consideration. 


A pool of 84 items discriminating at P <. 05 be- 
tween high and low groups as distinguished by scale 
Scores was obtained from the first two phasesof the 
adaptation. Inthe selection of 50 items for the third 
phase of the instrument's development, considera- 
tion was given to selecting items representative of 
the various propertiesof belief-disbelief systems as 
postulated by Rokeach. Items were assigned randomly 
to positions in the scale. 


FINDINGS 


The 50-item scale (Appendix) was administered 
by its developer to 485 pupils in all 17 sections of 
grades four through six in all five schools in a city 
of about 7,000 population in central Indiana. Thirty- 
four pupils were absent. Four of the five schools 
were public and one was Parochial. The sample popu- 
lation, once again, was described as being predom- 
inantly white, middle class, Protestant, native born 
and of northern European descent. | | 


Data on 27 of {һе 485 i imi 
Siete epee pupils were eliminatedfrom 


of the study; 2 j 
ers to be unable to read the i сате іасвеа ту кас 


economic scale no: 
ne 
teachers to be їп pe the 


records, the mea; 
population was fo nt of the sample 


range of 75 to 


135 and a standard deviation of ll. The average 
achievement grade level of pupils for selected school 
subjects was 5.3, with a range of from 2.3 to 9.3. 
Pupils’ scores on the instrument being developed 
ranged from 74 to 140, witha meanof 108 anda stand- 
ard deviation of 10. The distribution of scores did 
not vary significantly from normal. 


The median time taken by pupils to respond to the 
instrument was about 15 minutes and the range was 
from about 10 minutes to about 30 minutes. Odd-even 
reliability coefficients, as corrected for scale length, 
were found to be . 73 for grade four, . 60 for grade 
five, . 70 for grade six, and. 67for the total sample. 
Responses to individual items by pupils in high and 
low 27 percents of the total distribution of scores 
were compared through use of achi-square procedure 
and 48 items were found to discriminate at P <. 01 
and two (numbers 10 апа 17) were found to discrimi- 
nate at P <, 05. 


Scores attained by pupilsonthe revised instrument 
were compared by grade, socio-economic class, and 
School groupings by use of a critical ratio technique. 
The mean of scores for grade four (110. 64) was found 
to be significantly higher (Р <. 01) than the means 
for both grades five and six. The mean for grade five 
(107. 53) was higher than the mean for grade six 
(106. 68), but not significantly so. Means by socio- 
economic class did not differ significantly but there 


was a tendency towardan inverse relationship. The 
mean for pupils attending the parochial school was 
significantly higher (Р <, 05) than for two of the 
four public schools and the mean for pupils attending 
a public school in a low socio- economic area of the 
city was significantly higher than the mean for pupils 
in a higher socio-economic section of the city. 


Point biserial or product moment correlations 
were used to compare pupils' scores on the instru- 
ment and their sexes, ages, intelligence quotients, 
and achievement test scores. Allcoefficients were 
found to be іп thechance range ( -. 20 to +. 20 ) and 
none reached the P <. 05 level of significance. 


DISCUSSION AND CONCLUSIONS 


Some of the data, it was concluded, could be in- 
terpreted as evidence of the instrument's validity. 
Assuming a relationship between grade in school and 
amount of knowledge possessed, the inverse order 
of means of scores by grade groups would seem to 
support Rokeach's theoretical postulation concerning 
а positive relationshipbetween amount of knowledge 
possessed and openmindedness (5:38). Inagreement 
with findings of other studies (1:572 ), there were 
tendencies, though not consistent or significant, for 
Pupils in higher socio-economic classes to score 
lower than pupils inlower socio-economic 
classes. The finding concerning the higher mean 
for pupils in the parochial school was consistent with 
findings by Rokeach and others in studies of adult 
Populations (5:117-119). None of the data, it was con- 


cluded, could be interpreted as evide 
3 nce that ld 
tend to invalidate the instrument, ‚ к 


_ For the Sample population 
instrument was functioning г 
а measuring device and was 
Same facets of openmindedn: 


; it was concluded, the 
elatively effectively as 
measuring some of the 
ess-closedmindedness 
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among children that adult forms of the instrument 3. Figert, Russell L. Jr. An Elementary School 
measure among adults. The instrument, it was fur- Form of the Dogmatism Scale, unpublished doc- 
ther concluded, should be given additional trials with toral dissertation (Muncie, Indiana: Ball State 
sample populations having socio- economic,religious, University Library, 1965). 


and ethnic backgrounds varying from those of this 
study. It should also be used to select groups of high 


and low scorers for scale-validation studies using 4. Pannes, Ernestine D. An Exploratory Study of 
some of the techniques developed by Rokeach and the Relationship Between Attitudes Toward 
others. Finally, resultsof studies should be analyzed Self and the Degree of Dogmatism Present in 
for implications concerning the organization and pre- a Group of Junior-Senior High School Students, 
sentation of instructional materials. unpublished doctoral dissertation ( New York 
University, 1962). 
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APPENDIX 


Instructions and Items for the Elementary School 
Form of the Dogmatism Scale 


WHAT DO YOU THINK ? 


This is a way to find out how boys and girls think about things at different ages and in different grades. 
Boys and girls in grades four, five, and six are being asked to mark answers on what they think about the 50 
Statements on the next three pages. 

The statements are what boys and girls might think or say about things. You may find that most of the 
statements say things in the way you would. Or you may find that only a few of the statements Say things in the 
way you would. In any case, you will find that many boys and girls mark the statements in the same way you 
do. The answers you mark should be what you think about things. 


The answers you give are Secret. Do not put your name on your paper. Just put the grade you are in, 


the date of your birthday, if you are a boy or a girl, and how many years old you are, 


I am in grade . Iam years old. Iam a boy o girl EJ 


My birthday is on 


(month) (day) 


Here is how to mark the statements: 


l. If you find that a statement says what you think is so all of the time, make a circle 
around the plus sign (+) to the right of it. 


+ 0 is 
2. If you find that a statement does not say what you think is 50, make a circle around 
the minus sign (-) to the right of it. . А А 
3. 1f you find that a statement says what you think is so some of the time, and does not 
Say what you think is so some of the time, make a circle around the zero (0) to the 
right of it. " 0 _ 


Please make sure you mark one answer, and only one answer for each statement. 
it, but most boys and girls are able to finish in about 30 minutes. If you need help with a; 
your hand. Now go ahead with the statements. 


There is no time lim- 
ny of the WOrds, raise 


v 
RJ 


ДА 


21. 


22. 


23, 


24, 


25. 
26. 
27. 
28. 


29. 
30. 
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Some people are always for what is true. All the rest are always against what is 
true. 


We must believe what important people say. If we do not, we will not know what 
is going on in the world. 


Sometimes I start talking before someone else is done because I must tell what I 
have to say. 


I just cannot stop when I talk with people who do not see things like I do. 
Most people just do not care about others. 
It is not good to read books that get your thoughts about things mixed up. 


It is wrong to give in any to those who think different people should run our 
country. 


Some ways of thinking are really the same even though people try to tell you 
they are different. 


There is no use to take time to listen to someone who will just try to change 
your mind. 


People know more about things they believe in than things they are against. 


In times like these, people are selfish if they think first about their own hap- 
piness. 


I must make myself do a lot of things that I need to do. 

By saying things over and over you can be sure people know what you mean. 
Many things in books are not worth being there. 

People who think about themselves first are terrible. 

Most people seem smart enough. Even so, groups do dumb things at times. 
There is so much to do and so little time to do it in. 

Most people just do not know what is good for them. 

People must take chances if they really want to do big things. 


It Seems like many people I talk to do not really know about the good and bad 
things that are going on in the world. 


It does not matter much if 


you are not ha now. Iti h i i 
years to come that counts. not happy is what will happen in 


Most people seem to have odd ideas about a lot of things. 


When people are mad the ri 
1 > ойе 1 
| à y à n bring up things that do not have much to do 


It is bad to make 


eople а i 
World. people do things. 


Yet, often this is the only way to help the 


Iam sure people talk about me. 


Sometime 
8 we must make people do things we know are good for them 
Peo i 
Ple who will never Say they are wrong make me mad. 


Iti 
is better to be a dead hero than a live “chicken. " 
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31. Many times I do not listen to what people are saying because I am thinking 


of what I will say next. E 0 z 
32. People who do not believe in something important do not have much ofa 

life. + 0 = 
33. People get the most out of life only when they try hard to do what they 

think is best. + 0 S 
34. I would like it if someone would tell me how to take care of my problems. * 0 - 
35. People do not say much about it, but they are thinking about how they can 

become great. * 0 = 
36. We have a good way of running our country. Even so, it would be better if 

we let only smart people do it. + 0 - 
37. Most people seem to find things wrong with what others think. + 0 - 
38. If people knew what I really thought, they might not like me. + 0 - 
39. It is best to find out what smart people say about something before you say 

anything yourself. * 0 - 
40. If people had the chance, they would do Something great to help the world. * 0 "i 
4l. People seem to think that most of the things they do are bad. + 0 2 
42. Things are really bad when someone talks against people who think the same 

as he does. ж 0 - 
43. We are going against our own side if we listen to what the other side says. * 0 - 
44. Life is terrible if people do not have help. * 0 - 
45. People should not try to work together if they believe different things. * 0 - 
46. It would be better if we could go back to the way things were in the good old 

days. Е: 9 = 
47. It seems like people I do not even know are always looking for the bad things 

I — * 0 = 

in me. 
48. There are many ways to think about things in this world. Even so, there is 

only one right way. * 0 ^ 
49. Attimes I think I am no good at all. * 0 = 
50. Icannot stand some people because of the way they think about things. + 0 E 


Before you hand in your paper, please be sure you have marked one answer, and only one answer 
for each statement. 


THANK YOU:! 
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ABSTRACT 


The helping relationship experience was explored as a means of insuring greater rapport with children. A 
sample of 82 junior and senior education students participated for a minimum of two hours/week for approx j- 
mately eight weeks. Of this group, some were involved with white children; another group with Negro children; 
and a third group served as control. Groups were pre- and post-tested on selected personality variables felt to 
be necessary for the establishment of good rapport. A complex analysis of variance design was used in analyz- 
ing the data. Females and ‘‘liberals’’ scored significantly higher on the MTAI than males or “‘traditionalists. i 
Failure to find other evidence of a significant effect on the experimental groups was attributed to insufficient du- 


ration and/or intensity of the treatment. 


IN THE National Advisory Council's reporttothe 
President on the Education of the Disadvantaged Chil- 
dren, November 30, 1966, following an extensive 
summary of 86 school districts in 43 states, the 
Council abstracted six main conclusions, one of which 
dealt directly with rapport. The factor most fre- 
quently observed that distinguished favorable class- 
rooms from poor was the quality of the relationship 
between the teacher and the child (4). This would 
seem to underscore an urgent need for a new look 
at our present teacher education system. It may be 
that the current list of assumed characteristics felt 
to be necessary for an effective teacher, ascurrent- 
ly demonstrated by the curriculum requirements in 
our teacher education institutions, is incomplete 
and/or inappropriate, 


| It was concluded in an earlier study by one of the 
investigators that the student-teacher’s lack of sen- 
sitivity to children inhibited his attempt to commun- 
icate with them (3). Students subsequently were en- 
Couraged to interact with a child in a helping rela- 
tionship (9) for a minimum of two hours per week. 
Bon the student, child, and parent-teacher re- 
nsns to this experience were h ighly favorable. 
parents and teachers without exce ption ex- 
ш а sincere desire that the program be con- 
ue ER UE the students stated thatthe tutoring 
RM E tionship experience had beenthefirst 


i -confidence and self-es- 
t of this finding, others have sug- 


n For thi i А 
first became inter, 4118 reason the investigators 


helping relationship with children asa rather unique 
experience in teacher education. 


THE PROBLEM AND PURPOSE OF THE STUDY 


An assumption of this study is that our teacher 
education programs, in general, are presently not 
designed to adequately develop and strengthen t he 
personality characteristics needed to insure the fu- 
ture establishment of good rapport between the stu- 
dent and his pupils. It may be agreedthat openness, 
sensitivity, receptivity, compassion, understanding, 
etc. are personality traits most frequently found in 
individuals demonstrating the ability to establish 
good rapport. The problem is one of discovering 
means for effectively developing and strengthening 
Such personality traits. The purpose of this study 
was to explore the effects of a helping relationship 
experience on selected personality variables felt to 
be significant in the establishment of good rapport. 


RESEARCH HYPOTHESES 


Hypothesis I Тһе involvement of a student with 
a child ina helping relationship en- 
hances the student's general feel- 
ing and sensitivity for children. 

Hypothesis П The involvement of a student with 
a child in a helping relationshipen- 
hances the student’s feelings about 
himself. 

Hypothesis III The involvement of a student with 
a child ina helping relationship sig- 
nificantly affects the ‘‘dogmatism- 
liberal’’ characteristics of his be- 

| lief system. 

Hypothesis IV Students in a helping relationship 
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with Negro children tend to view 
themselves and the Negroasbeing 
more alike than students who have 
not tutored Negroes. 

Hypothesis V Student performance on the Minne- 
Sota Teacher Attitude Inventory is 
significantly affected by the degree 
to which the student is 1) **dog- 
matic’’ and 2) secure." 


DESIGN OF THE STUDY 


Sample 


Three teachers of a child study unit at Eastern 
Kentucky University volunteered their students as 
subjects for this research. All of the students were 
either juniors or seniors enrolled in the teacher ed- 
ucation program during the 1967 spring semester. 
The sample involved 32 white males and 50 white 
females. 


Procedure 


The three teachers who volunteeredtheir students 
for this study did not completely agree on the value 
of the helping relationship experience. It was on 
this disagreement that the researchers attempted to 
capitalize. Two of the teachers felt such an experi- 
ence should definitely be offered their students; how- 
ever, only one made it a requirement. Consequently, 
in the case where it was optional, а few students 
(three males and seven females) chosean experience 
other than working with a child. The third teacher 
did not provide for the helping relationship ex- 
perience, 


The initiative of the students who participated in 
the helping relationship and the availability of the 
Children were probably the greatest determiners as 
to the particular age, sex, and race of a child. Con- 
sequently, the researchers initially were not sure 
as to whether the subjects would distribute them- 
selves adequately for analysis. The statistical de - 
Sign was thus dependent to a large degree upon the 
Student's choice at the beginning of the semester and 
the availability of the children. It was the ho pe of 
the investigators that each subject be classifed as 
having been involved in a helping relationship with 
either a Negro child, a white child, or none atall. 
The students actually distributed themselves across 
the three categories in the following manner: — 

1) Thirty-four Ss did not Lene ыш in a helping 

relationship, the control group. | | 

2) Sixteen Ss Шр into a helping relationship 

with a Negro child, an experimental group. - 

3) Thirty-two Ss entered into a helping re peu 

Ship with a white child, an experimental group. 


d during the 
prior to the ini- 
ted the last three 


_ All of the subjects were pre-teste 
first three weeks of the spring “en 
lation of the treatment and post-tes | ; 
Weeks following the treatment. Four ee = 
explored in investigating the effects of ihe en $ 

relationship experience, l1)sensitivity to С et 

Seneral security, 3) dogmatism, and 4) prej . 
i e 

The instruments used to investigate each of th 
Variables are listed below: 


3 
- Minnesota Teacher's Attitude Inventory 


Maslow's Security-Index Inventory 
Rokeach's Dogmatism Scale 

Bogardus Social Distance Scale (Appendix A) 
A Trait Evaluation Scale (Appendix B). 


yews 


Unfortunately the MTAI was not available for pre- 
testing. It should be recognized, therefore, that any 
differences found in performance on the MTAI be- 
tween the experimental and control groups may have 
existed prior to the treatment. The other tests, 
however, were available from the beginning. Con- 
sequently, differences which have taken into account 
the relative position of each group from the start 
can more reasonably be attributed to the effects of 
the treatment. 


The Treatment 


The treatment consisted of a helping relationship 
experience (9) with a child for a minimum of two 
hours per week for ten weeks. The child involved 
in this experience was selected by the student. 
There were no limitations on exactage, sex or race, 
only that they be children. The situation (or condi- 
tion) under which this relationship could function 
was limited only by guardian approval andthe safety 
of the child. Most of the students in the experimen- 
tal group found themselves either actingas a big 
brother, a baby sitter, or a tutor. Regardless of 
the situation, their purpose was all the same; to be- 
come a friend to the child with whom they were 
involved. This required a special effort on their 
part to become more open- minded, more accepting, 
more transparent, and more sensitive. It was ex- 
pected that some would be more successful than oth- 
ers. The important aspect of the experience, how- 
ever, was the fact that all of the students involved 
in a helping relationship were presented with an ex- 
cellent opportunity to promote their own personal 
adequacy. 


The Instruments 


The Minnesota Teacher Attitude Inventory (MTAI). 


This scale has successfully identified teachers cap- 
able of establishing good rapport with students. Rea- 
sonably high validity and reliability coefficients have 
been reported. The scale has been found to be only 
slightly susceptable to attempts to ''fake good" 
(Test Manual). 


Maslow's Security- Index Inventory (S-1 Inven- 
tory). The validity of this measure depends toa 
large extent upon the subject's willingness to dis- 
close himself. Consequently, the test was not ad- 
ministered until it was felt that an acceptable degree 
of confidence had been established between the pro- 
fessor and his students. Of those subjects randomly 
selected from the sample, none felt the interpreta- 
tion of their score to be invalid (Test Manual). 


Rokeach’s Dogmatism Scale (Form E). Five of 
the original statements of Form E were omitted 
(Nos. 11, 12, 13, 14, and 15 on Form D) from use 
in this study because of their similarity to the type 
used in Maslow’s S-I Inventory. As this lowered 
the number of items to 35, five suitable questions 
were chosen from: Form D (No. 4, 7, 29, 56, 
and 57) in order that the sample means be com- 
parable with the norms (10). 
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The subjects were asked to indicate the extent to 
which they agreed or disagreed with each statement 
on a scale ranging from one to seven. The state- 
ments were all of a dogmatic nature so that strong 
agreement — 7 — would be interpreted asa dogmatic 
response and strong disagreement - 1 - as a liber- 
alresponse. The theoretical mean was 160. 


Social Distance Scale. The scale was made up of 
statements selected on an a priori basis to elicit 
responses indicative of the subject's degree of ac - 
ceptance of the person being considered (Appendix 
A). Reviewers report that it has been widely and 
successfully used for this purpose (6). 


Trait Evaluation Scale. The trait evaluation 
scale consisted of 24 trait dimensions (Appendix B). 
Each dimension was represented by two diametri- 
cally opposed traits separated by a seven-point con- 
tinuum, Subjects were asked to rate so meone or 
themselves on each of the 24 dimensions. The aver- 
age of their total ratings yielded a theoretical mean 
position in space. Logically, it would seem that 
people can judge others only in relation to them- 
selves, and furthermore, that those judged as more 
similar are generally perceived as more acceptable. 
With this in mind, subjects were asked to rate them- 
Selves and another. Thedifference between 
these scores was assumed to be representative of a 


theoretical distance between the subject and the oth- 
er person. 


Statistical Analysis 


As the purpose of the study was to determine the 
effect on students in teacher education of a helping 
relationship experience, the statistical design was 
oriented toward exploring differences between 
groups in degree of performance c hange on tests 
related to selected personality variables. Differ- 
ences between pre- and post-test performances 
were computed. As this difference Score was sub- 
Ject to a plus or minus Sign, a suitableconstant was 
added. The results were then subjected to a com - 
plex analysis of variance to investigate possible dif- 
ferences in changes in performance between various 
groups due to the treatment (8. This procedure 
was appropriate for all the variables exce pt that 
measured by the Minnesota Teachers Attitude Inven- 
tory. As this test was not available for pre-testing, 
the results obtained at the termination of the treat- 
ment could not as readily be ascribed to the effect 
of the treatment. 


RESULTS 


Coefficients of reliability were obtained on 
Struments used in this study by computing a Pear- 
son І correlation between pre- and post-test. The 
findings are shown in Table 1. 


in the 


| All of the measu 
with the exception of t 


As the relationshi 


р between the Во 
and other similar m gardus Scale 


easures was of interest to the 


o AREE 


TABLE 1 


RELIABILITY COEFFICIENTS USING TEST- 
RETEST METHOD 


Source N 


Txx 
S-I Inventory 82 .88 
Dogmatism Scale 82 Т? 
Bogardus Social Distance 
Scale on Negro 82 JT 
Trait Evaluation Scale 
on Negro 82 . 76 
Trait Evaluation Scale 
on Self 82 ‚51 


All correlations listed are significant at or beyond 
the . 01 level. 


researcher, Pearson r's were computed between 
this scale and two adaptations of the Trait Evalua- 
tion Scale, The first adaptation of the scale ex- 
plored the relationship between howa subject evalu- 
ated Negroes and his social acceptance of Negroes 
as measured by the Bogardus Scale. The second 
employ ment explored the relationship between 1) ће 
differences between how a subject evaluated himself 
and how he evaluated the Negro, and 2) the Bogardus 
Scale. The results of these findings are listed in 
Table 2. 


TABLE 2 


CORRELATIONS BETWEEN BOGARDUS SCALE 
AND SIMILAR MEASURES ON THE NEGRO 


Source N 


гуу 
Trait Evaluation Scale 

(Negro only) 82 .40* 
Trait Evaluation Scales 

(Selt- Negro) 82 .43* 
*Pcm 01 


The correlations between a subject's social ac - 
ceptance of the Negro (Bogardus Scale) andhis eval- 
uation of him (Trait Evaluation Scale), both sepa- 
rately and with regard to himself, were surprising- 
ly low (Table 2). 


The statistical method of analysis of variance 
was used to estimate the degree to which change 
may have been involved in group differences. This 
procedure was followed as a method of testing the 
formal hypotheses. The first hy pothesis questioned 
the effect of the treatment upon the subject’s sensi- 
tivity to children. The MTAI Was used to measure 
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this variable. Asit wasnotavailable for pre-testing, 
the analysis between groups could be made only on 
the post-test. Figure 1 illustrates the group means 
by sex and treatment. 


FIGURE 1 


GRAPHED MEANS OF MTAI BY SEX AND 
TREATMENT (N - 82) 


Treatment 


Mean Tutored White Tutored Negro Control 


i00[ | | 


Qe 
90 | unc Son 
80| — 3 


70 


60 


|j 


Note: Constant of 50 added to all scores. High 
mean scores denote greater “sensitivity. " 


The results of the MTAI analysis of variance by 
Sex and treatment reveal sex to be the only signifi- 
cant factor (Table 3). Female education students 
tend to score higher. 


The second hypothesis was concerned with the 
effect of the helping relationship upon the subject's 
own feelings of personal and socialadjustment. Two 
analysis of variance computations were undertaken. 
The first sought to discover whether, in fact, there 
was a difference between the experimental groups’ 
pre- and post-tests. The performance means of 
the S-I Inventory by sex and time of testing are il- 
lustrated in Figure 2. 


The analysis of variance did not reveal any sig- 
nificant differences due to the two independent var- 
iables (Table 4). 


The second analysis of variance investigated per- 
formance differences on the S-I Inventory by sex 
and treatment. The means for this analysis are il- 
lustrated in Figure 3. 


The performance scores for computing the anal- 
ysis of variance of the S-I Inventory by sex а nd 
treatment were obtained by subtracting the post- 
test from the pre-test and adding a suitable con- 
Stant to eliminate negatives. Computed differences 
were, therefore, differences in change. 


The analysis of variance did not reveal any sig- 
nificant differences in change due to the two inde- 
pendent variables (Table 5). 


FIGURE 2 


GRAPHED MEANS OF S-I INVENTORY BY SEX 
AND TEST (N - 96) 


Test 


Mean Pre-Test Post-Test 


| 


Note: Low scores denote greater “security. "' 


The third hypothesis was concerned with the ef- 
fect of the helping relationship upon the subject's 
cognitive flexibility as measured by the Dogmatism 
Scale. The procedure of analysis for the scale was 
identical with that employed for the S-I Inventory. 
Two analyses were undertaken. The performance 
means achieved on the Dogmatism Scale by sex and 
time of testing are illustrated in Figure 4. 


FIGURE 3 


GRAPHED MEANS OF S-I INVENTORY BY SEX 
AND TREATMENT (N - 82) 


Mean Tutored White Tutored Ne ^ 
$6 [— egro Control 
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52 
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Note: Constant of 50 а; 
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ward security, 
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iffer 
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enote change to- 
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TABLE 3 


MTAI ANALYSIS OF VARIANCE BY SEX AND TREATMENT (N - 82) 


Source 


Sum of Squares df Variance Est. Е 
Between Cells 
Experimental-Control (rows) 1,010. 57 2 505. 29 N.S. 
Sex (columns) 4,716. 69 1 4, 116. 69 5. 93 
Interaction 41. 17 2 20.59 N. S. 
Within Cells 60, 169. 48 76 794. 75 
Total 66, 169. 48 81 
*Р = .05 
TABLE 4 
S-I INVENTORY ANALYSIS OF VARIANCE BY SEX AND TEST (N = 96) 
Source Sum of Squares df Variance Est. F 
Between Cells 
Pre- Post Test (rows) 2.34 1 2.34 N.S, 
Sex (columns) 19. 63 1 19. 63 N.S. 
Interaction 17. 53 1 17.53 N.S. 
Within Cells 17,024. 99 92 185. 05 
Total 17,065. 49 95 
TABLE 5 
S-I INVENTORY OF ANALYSIS OF VARIANCE BY SEX AND TREATMENT (N = 82) 
Source 
Sum of Squares df Variance Est. F 
Between Cells 
Treatment (rows 
Sex (columns) 5) ps 2 50. 44 N.S. 
Interaction EN 1 92. 20 N.S. 
е xn 2 6.91 N.S. 
ithin Cells 
ne 2,643. 21 16 34.78 
Total 
2,850. 10 81 


mm 
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FIGURE 4 


GRAPHED MEANS OF DOGMATISM SCALE BY 
SEX AND TEST (N= 96) 


Test 


Post- Test 


134 


Note: Low scores denote liberalism. 


The analysis of variance for the Dogmatism Scale 
by sex and test did not reveal any significant differ- 
ences due to the two independent variables (Table 6). 


The second analysis of variance having todo with 
the dogmatism variable investigated performance 
differences by sex and treatment. The means for 
this analysis are illustrated in Figure 5. 


The analysis of variance for the Dogmatism Scale 
by sex and treatment revealed no significant eigen" 
ences in change due to either of the independent var- 
iables (Table 7), 


The fourth hypothesis explored the question of 
the effect of the helping relationship upon the sub- 
ject's evaluation of the Negro in relation to his eval- 
uation of himself. To accomplish this, each sub- 
ject's evaluation of the Negro was subtracted from 
his trait evaluation of himself for both pre- and post- 
test differences. The result wasa score represent- 
ative of the degree of change in the way each subject 
evaluated the Negrorelative to himself, i.e., 

Self, -Negro,) - (Self, -Negro;)]. 


High scores indicated greater closure between 
the Self-image and the Negro-image. The means of 
the converted scores from the Trait Evaluation 
Score are recorded in Figure 6 by sexand treatment. 


The analysis of variance gave no indication of any 
Significant effect upon image closure due to the 
treatment or sex (Table 8). 


The final hypothesis of this study concerned the 
effect of **dogmatism" and ‘‘security’’ on groupper- 
formance on the MTAI. 


The entire sample of subjects was divided into 
three groups by their performance on the Dogmatism 
Scale and three groups by their performance on the 
S-IInventory. The middle £roup in both cases con- 
sisted of those subjects within a half Standard devi- 
ation of the sample mean. 


The performance means of the MTAI by ‘‘dogma- 
tism"' and "security" are illustrated in Figure 7. 


The MTAI analysis of variance by security and 
dogmatism revealed a very significant difference be- 
tween dogmatic and liberal groups in their perform- 
ance on the MTAI (Table 9). 


FIGURE 5 


GRAPHED MEANS OF DOGMATISM SCALE BY 
SEX AND TREATMENT (N = 82) 


Treatment 


Mean  Tutored White Tutored Negro Control 
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TABLE 6 


DOGMATISM ANALYSIS OF VARIANCE BY SEX AND TEST (N - 96) 


Source Sum of Squares df Variance Est. F 
Between Cells 
Pre- Post Test (rows) 372. 09 1 372. 09 N.S. 
Sex (columns) 70. 20 1 10. 20 N.S. 
Interaction 105. 39 1 105.39 N.S 
Within Cells 51,858.56 92 563. 68 t 
Total 52, 406. 24 95 


TABLE 7 

DOGMATISM ANALYSIS OF VARIANCE BY SEX AND TREATMENT (N - 82) 

Source Sum of Squares df Variance Est. F 

Between Cells 
Treatment (rows) 310. 40 2 155. 20 N.S. 
Sex (columns) 251.83 1 251. 83 N.S. 
Interaction 132. 71 2 66. 36 N.S 

Within Cells 15,331. 84 76 201. 73 

Total 16,026. 78 81 

TABLE 8 


MENT 
So 
urce Sum of Squares df Variance Est. F 
Between Cells 
Treatment (rows) 
Se 19, 793, 65 2 3 
im 2,304. 97 1 ? a or ^ > 
8,595. 90 2 4, 297. 95 E 
Withi | ` 
thin Cells 391, 678. 41 16 5, 153. 66 
Total ` 
А 422,372, 93 81 
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FIGURE 6 


GRAPHED MEANS OF IMAGF CLOSURE BETWEEN 
SELF AND NEGRO BY SEX AND TREATMENT 
(N = 82) 


Mean.  Tutored White Tutored Negro Control 
350 | | 


340] — d 
330 
320 
310 


300 


290 » 
280 


270 


Note: Constant of 300 added to all difference Scores 
[05 1-№) - (S2-N2)]. Mean scores above 300 de- 
note Image Closure. 


“Liberal” groups, as illustrated in Figure 7, 
tend to score higher on the MTAI than “‘dogmatic’’ 
Бгоирѕ. These differences appear to diminish a s 
subjects move toward a state of insecurity. The dif- 
ferences due to extent of security, however, were 
not statistically significant. 


CONCLUSION AND DISCUSSION 


The statistical procedures for analyzing the data 
included correlation analysis and complex analysis 
of variance. All tests of significance were two- 
tailed. From the relatively high correlation 5 


FIGURE 7 


GRAPHED MEANS OF MTAI BY DOGMATISM 
AND SECURITY (N - 82) 


Mean Secure Average Insecure 
110| — | 
(Liberal) | 
100| — | 
90| — | 
80| (Average) X 
70| — 
(Dogmatic) e ы 
60 


її] 


Note: High scores denote greater sensitivity to- 
ward children. 


achieved between the pre- and post-tests, it may be 
concluded that the stability of the instruments was 
adequate for the purpose of this study. Therela- 
tively low correlation between pre- and post-tests 
for the Trait Evaluation of Self in comparison with 
the correlation for the Trait Evaluation of the Negro 
was interesting. It may be that a person's beliefs 
about himself are subject to more variance than his 
beliefs concerning others. Whether this isonly true 
of teacher education students with regard to Negroes 
is uncertain. It does, however, point up a signifi- 
cant degree of uncertainty and/or a Striking amount 
of change in the self-concept at this stage. 


The second use of correlation analysis involved 
the Bogardus Social Distance Scale with another 
measure felt to be similar in nature. The obvious 
intent of the Bogardus Scale is to explore the degree 
to which a subject is willing to accept a pre-selec- 
ted individual or type socially. The Trait Evalua- 
tion Scale was intended to reveal the subject’s feel- 
ings for the individual, both independently and rela- 
tive to the subject's own evaluation of himself. The 
low correlations indicated a poor relationship be- 
tween the degree to which subjects are willing to ac- 
cept another subject and the way they evaluate him 
аз а person. Evidently, regard for an individual is 
not a clear indication of acceptance. It may be that 
the Trait Evaluation Scale fell short of specific 
areas related to acceptance. Another possibility is 
that responses on the Trait Evaluation Scale are 
simply not representative of what might be expected 
Írom the Bogardus Scale due to the subject'sown un- 
awareness of his true feelings. In such a case, the 
subject may be quite willing to recognize desirable 
qualities in another person but unable to accept him 
socially. Recognition of an individual as being sim- 
ilar to one's self, evidently, does not ass ure his 
social acceptance. 


Only one of the five hypotheses was Supported to 
any degree by this investigation. The treatment was 
apparently not of sufficient type or duration to s ig- 
nificantly affect the variables under study. Although 
the differences in means were often in the antici- 
pated direction, the variance within groups wa S of 
Such magnitude that the results could just as easily 
have been attributed to chance. 


| The analysis of variance revealed only 
nificant F's, neither having to do with the 
relationship. " In the male 
MTAI 
To the extent that the MTAI 
ubject's ability to co 
; it may beconc luded 
male education Students, in most cases, Whee 


in the classroom, 


The other significant findin 
to the last hypothesis, 
h = = А s 
pothesis, the | to explore the 
on the sub- 


that a subjects gener, i indicated 
ject al feeling of sec 
Meu his Sensitivity to Children, c) mus little 
У, however, does n ive 
factor ‘Liberal a Bear d 


be a significant 
Significantly 


E _` 
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TABLE 9 


MTAI ANALYSIS OF VARIANCE BY SECURITY AND DOGMATISM (N - 82) 


Source 


Sum of Squares df Variance Est. E 
Between Cells А | 
Security (rows) 3,033. 26 2 1,516. 63 N.S. 
Dogmatism (columns) 9,558.55 2 4, 119. 28 6. 50* 
Interaction 1,106. 87 4 2,767. 21 N.S. 
Within Cells 53,361.77 73 735. 11 
Total 67,361.77 81 


*P = .005 


higher on the MTAI than ‘‘dogmatic’’subjects. It 
may be that the variable needing the most attention 
in teacher education is that of ‘‘open- mindedness. '' 
Such an interpretation seems theoretically reason- 
able when one considers education to be a process 
of communication. The teacherasboth ‘‘receiver”’ 
and ‘‘sender’’ of information must have some aware- 
ness of how the child perceives a problem. Such an 
awareness on the part of the teacher tends to insure 
the transmission of information relevant to the 
child's own level of awareness. An effective teach- 
er must, therefore, have the ability to constantly 
readapt psychologically to the ever changing per- 
ceptions of the child. Such demands made of a 
“rigid’’ or ‘‘dogmatic’’ individual would most cer- 
taily result in anxiety and frustration. 


The Morrison and Romoser factor analytic study 
of the MTAI represents evidence in support of this 
theory (7). Their findings indicate that students 
who performed poorly on the MTAI tended to be 1) 
traditional in their beliefs, 2) anxious, 3) less in- 
telligent, and 4) male. 


__With regard to the present study, no significant 
differences were found between groups due to anxi- 
ety (S-I Inventory). Groups classified as either 


“female” or ‘liberal’ did, h 1 з 
on the МТАТ, id, however, score higher 


| It would seem, therefore, that the 

у iable measured by the MTAI is closely allied to 

Philosophical or cognitive beliefs concerning ‘‘liber- 
Sm.’’ Why a subject res ponds *'traditionally, '" 


however, on the MTAI may coi i 
, 
: Г y conceivably result from 


It is 
of traditionalism AN doubtful that the effect 


Subject's sensitivity to chil- 
dren would be the Same irrespective of cause. 


Á arch inthearea. The 
esas Che еши inability of the type of cur- 


The fact that such vari 
di riablesare beli 
irectly related to the establishment of ve Ый 


lends additional weight to the findings. Future in- 
vestigations into the question of how teacher educa- 
tion can increase its effect upon its students should 
not be limited to the area of modification alone, as 
demonstrated in this study, but should also consider 
the effects of duration and intensification. 


FOOTNOTES 


1. The writers wish to acknowledge the services 
and assistance of many persons who con- 
tributed to this study. They are especially 
appreciative of the help given to them by Mr. 
Don Mills, Dr. Elizabeth Parker, Miss 
Sharon Congleton, their secretary, and the 


participating students, allof Eastern Kentucky 
University. 


2. The work presented or reported herein was per- 
formed pursuant to a grant from the U. S. Of- 
fice of Education, Department of Health, Ed- 
ucation, and Welfare. 


3. This test was not available for pre-testing. 
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APPENDIX A 
Social Distance Scale 


Date 
——— 


Name 
Score 
Type of Individual Being Considered 


——À 


The instructions for the scale are as follows: 
*According to my first feeling reactions, I would 
willingly admit the type of individual considered on 
the S-D Scale No. 1to one or more of the classifi- 
cations under which I have placed a cross, ” 


1. To close kinship by marriage. 

2. To my club as a personal chum. 

3. To my street as a neighbor. 

4. To equal employment in my Occupation. 
5. To citizenship in my country. 

6. Asa visitor in my country. 

T, To citizenship in a foreign country. 

8. As a member of the human race. 


APPENDIX B 


Trait Evaluation Scale 


Date Name Score 
Individual Being Considered 
Score 

1, Progressive 1234567 Conservative ae 
2. Honest 1234567  Dishonest PEE ын 
3. Physically clean 1234567 Physically dirty hers, 
4. Respectable 1234567 Indecent aaa. 
5. Reliable 1234567 Unreliable Se 
6. Compassionate, warm-hearted 1234567 Unforgiving, unmerciful —_ E — 
7. Intelligent 1234567 Unaware, ignorant ee 
8. Loyal 1234567  Unfaithtul a шы 
9. Industrious, hard-working 1234567 Lazy, indolent ee 
10. Practical, sensible 1234567 Foolish, irrational Lees 
1l. Creative, enterprising 1234567 Unimaginative, traditional m 
12. Alert 1234567 Dull, slow Te 
13. Courteous, polite 1234567 Bad- mannered, boorish, rude ~~~ 
14. Humorous 1234567 Critical, solemn, grim aa 
15. Ambitious, eager 1234567 Unmotivated, indifferent SS 
16. Persistent, persevering 1234567 Yielding, submissive n 
17. Friendly 1234567 Hostile, quarrelsome omen ы“ 
18. Altruistic (likes people) 1234567 Misanthropic (hates people) ita. 
19. Productive 1234567 Useless, ineffective See 
20. Likeable, enjoyable 1234567 Obnoxious, offensive MEME E 
21. Scientific 1234567 Superstitious Еа. 
22. Attractive 1234567 Repulsive lm c 
23. Optimistic 1234567 Pessimistic T 
24. Realist 1234567 Idealist in 00 

a ee 
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AND READING READINESS 


MINNEAPOLIS PUBLIC SCHOOLS 


PATRICIA J. GORALSKI and JOYCE M. KERL 


ABSTRACT 


In order to discover whether teacher aides can be use 


dergarten children, three groups of three classes e 


(Metropolitan Readiness Test, Form В), 
favor of the groups with aides. 


TEACHER AIDES have served the Minneapolis 
Public Schools in increasing numbers since the sum- 
mer of 1965. During the school year, 1966-67, 260 
teacher aides worked in Minneapolis target area 
Schools. Periodic evaluations have indicated that 
their work has been well received by both educators 
and the community. The aides have been enthusiastic 


about their participation in the educational experi- 
ences of children. 


PURPOSE 


The first experimental attempt to discover a re- 
lationship between aides’ services and pupil gain was 
made during spring semester, 1967. Nine classes 
in four Minneapolis target area schools participated 
in an experiment which attempted to answer the ques- 
tion: “Can teacher aides be used effectively to help 


to develop reading readiness in kindergarten child- 
ren?” 


SAMPLE SELECTION C RITERIA 


Nine kindergarten classes taught by six teachers 
were identified which met the following criteria: 


--Each class con 


sisted of approximately thirt 
children, d ни 


--Approximate ly half of 


> the classes met inthe 
morning and half in th 


€ afternoon. 


7-None of the c 


i lasses had had thi 
aides during 


e services of 
the fall semester 


‚ 1966. 


Under the direction of the teachers, aid 
iness. There were no initial differences t 
but there were significant differences at the e 


d effectively to help develop reading readiness in kin- 
ach were compared: (1) GroupA, note: 
B, one teacher aide, (3) Group C, five teacher aides. 

children in activities designed to increase reading read 


heraide, (2) Group 
es worked with 
»etween groups 
nd of one semester in 


--Only classes from target area schools that did 
not have lunch programs were included. (Where 
lunch programs existed, afternoon kindergarten 
sessions were shorter). 

--Of the thirty children in each classroom, ap- 

proximately one-third had participated in Head- 

Start during the summer of 1966. (The reason 

for this criterion was to furnish data for long- 

range Headstart evaluation), 


--Three of the teachers were to have both morn- 


ing and afternoon classes which met the other 
Criteria, 


DESIGN AND TESTING PROCEDURE 


The null hypothesis was: 


uu There is no difference between Ne Уш 
of second semester kindergarten pu 
Pils in reading readiness mean gain 
Scores on the Metropolitan Readiness 
Test. 


A diagram of the design of the experiment has been 
Presented in Table 1 


archers wanted the strongest c on- 


TOUp with five teacher aides, six of 


Spr nstituted the morning and afternoon 
Each df гае teachers and made up Groups A and 
and five a. ese teachers had no aide in one class 

aides in the other. Teachers of Groups A 


effects for the group with no teacher 
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TABLE 1 


THE EXPERIMENTAL DESIGN 


Group A 
No teacher aide 


Group B 
One teacher aide 


Group C 
Five teacher aides 


Class 1, Teacher 1 
School W, P. M. 


Class 2, Teacher 2 
School X, A. M. 


Class 3, Teacher 3 
School X, A. M. 


Class 4, Teacher 4 
School Y, A. M. 


Class 5, Teacher 5 
School Y, P. M. 


Class 6, Teacher 6 
School Z, A. M. 


N=75 Total, Group A 


N-38M N-39 M 
OF 
=79 Total, Group B 


Class 7, Teacher 1 
School W, A. M. 


Class 8, Teacher 2 
School X, P. M. 


Class 9, Teacher 3 
School X, P. M. 


and C were careful to use the added assistance for 
Group C classes, only, so that effects would not be 
contaminated. Group B consisted of the classes of 
three different teachers with one aide assigned to 
each class. 


Further, teacher effects tended to cancel because 
only group effects were analyzed and each groupcon- 
sisted of three different teachers. However, the 
frase control for teacher effects was provided 
in that 


--pretests were not administered until the end of 
the first semester when teachers had already 
worked with their classes for half the year 


--there were no initial differences between groups 
nor was there interaction 


--the same teachers continued with their classes 
throughout the second semester. 


Reading readiness wasassessed by the Metropoli- 
tan Readiness Test, Form R, (1949). Pretests were 
administered to 254 children in the nineclasses dur- 
ing the period from January 25, 1967, through Feb- 
ruary 3, 1967. Posttests were administered to 248 
children during the periodfrom May 22, 1967, through 
May 26, 1967. Testing yielded 234 pairs of test 
scores, Aides served six of the nine classrooms 
from February 6, 1967, through June 2, 1967. 


TREATMENT AND RATIONALE 


It was hypothesized that a teacher would be likely 
to view one aide as an extension of herself, as an 
assistant to help her with whatever she was doing. 
It would not be necessary for the teacher withone 
aide to develop a concept of the role of theaide sep- 
arate from her own activity. 


It was further hypothesized that it would be impos- 
sible for teachers with five aides in one classroom 


to proceed without giving a great deal of thought to 
the kinds of things that aides might accomplish which 
would be in addition to, or different from, what she 
might accomplish alone. The design of the experi- 
ment made it necessary for the three teachers of 
Groups A and C to think about differences in the ap- 
proach to problems of teaching and learning where 
there was only the teacher in the classroom (Group 
A) and where there were five adults inadditionto the 
teacher (Group C). 


During two inservice meetings, teachers exchanged 
ideas about the effective use of aides. Teachers were 
encouraged to use whatever approaches they thought 
might be effective in achieving maximum read ing 
readiness gains with pupils. It wasemphasized that 
with more adults in the classroom, there could be 
almost total participation of all childrenat alltimes: 
More physical activity, more verbal ехрге ў 
interaction. The tendency of adults to d 
children rather than helping children to learn to d 
things for themselves was explored. However, eai i 
teacher was completely free to use the services ae 
her aides in any way that she believed to be best, 


ssion, more 
o things for 


It was an unsettling experience f 
w or a teacher 
add five adults to one classroom and classroor v 
ministration problems arose, However, as the = 
ester progressed, teachers discovered that th e de 
were important services that five adults coul о ч 
form in one classroom. Teachers expressed a qa 
ues in trying a similar experiment in the f а 
С of the teachers stated that she would lik ше. 
ауе all five of her aides again the following cul 
заг. 


" It would have been valuable 
ree teachers of Grou 2 
to record: "m 


andinte 


resting if 
d C had & U the 


kept a diary 


~-their feelings as five 
dergarten classroom 

ће problems that arose 
with children ; 

"techniques used to solve 


adults came into one kin- 


Botha mone adults and 


problems 
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--ways that they tried to use aides' services ef- 
fectively to increase reading readiness in child- 
ren , 

--areas where aides were successful in enhanc- 
ing pupil learning 

--areas where aides did not seem to be success- 
fulin helping pupils to advance their learning. 


However, since the addition of five adults to one kin- 
dergarten classroom wasa rather overwhelming pros- 
pect for any teacher, it was feared that added paper- 
work might change very interested and cooperative 
teachers into rather reluctant ones, Itis recommend- 
ed that future experimentation include provision for 
participants to record their feelings about working 
with adults in the classroomas wellastheirattempts 
to work through teaching and learning problems. A 
weekly tape-recorded meeting would be an effective 


technique for ascertaining both essence and 
fact. 


ANALYSIS AND INTERPRETATION OF 


THE 
DATA 


In order to make a meaningful study of the major 
hypothesis, it was necessary to find out whether, at 
the beginning of the study, groups differed with re- 
Spect to reading readiness characteristics. Since 
it has been widely held that there are differences in 


readiness by sex, the following sub hypothesis was 
tested. 


Hg There is no difference in pretest mean 
Scores on the Metropolitan Readiness 
Test between groups or by sex, 


Three analyses of variance were calculated, one for 


each of three aspects of the test. They were reported 
in Tables 2, 3, and 4, 


TABLE 2 


ANALYSIS OF VARIANCE, PRETEST SCORES ON 
THE METROPOLITAN READINESS TEST, FORMR 
TESTS 1-4, READING READINESS 


Source of Variation df ss MS F 
Groups 2 14, 22 Tii 7 
Sex 1 16.80 16.80 4.05 
Interaction 


2 4.58 2.29 0.55 
Error, Within Cells 228 36804. 75 


Critical Vay i 
"et Е, 95 (1,228) = 3, вав 


Р. 
95 (2, 228) = 3. 0367 


y Ot rela 3 
У teacher aides) 


TABLE 3 


ANALYSIS OF VARIANCE, PRETEST SCORES ON 
THE METROPOLOTAN READINESS TEST, FORM R 
TEST 5, NUMBER READINESS 


Source of Variation df SS MS F 


Groups 
Sex 
Interaction 


2 0.18 0.09 0.15 

il 

2 
Error, Within Cells 228 


1.59 1.59 2.56 
1.63 1.31 2.11 
5513.41 0. 62adjusted 


Critical Value: F,95 (1, 228) = 3.8851 
Е 95 (2,228) = 3. 0367 


4. 15,adjusted 


TABLE 4 


ANALYSIS OF VARIANCE, PRETEST SCORES ON 
THE METROPOLITAN READINESS TEST, FORM R 
TEST 1-6, TOTAL READINESS 


Source of Variation df SS MS Е 
Groups 2 17.41 8.71 0.99 
Sex 1 34.71 34.71 3. 96 
Interaction 2 15.43 7.72 0.88 
Error, within cells 228 77663. 96 


Critical Value: F, 95 (1,228) = 3, 8851 


F., 95 (2, 228) = 3.0361 


ences, by groups, at the 5 percent 
there were Significant difference 
level of confidence, by sex, in readin, i 

ә Е readiness and 
total readiness. Therefore analysis of covariance 
which allowed for initial diff t 


the major hypothesis: 


BH There is no difference between groups 
of second semester kindergarten pupils 
in reading readiness mean gain scores 


on the Metropolitan Readiness Test. 


Three analyses of covariance were calcul; 
ported in Tables 5, 6, and 7. ino andre 
It was found that, after initi 
adjusted by means 
were no significant sex 
Cent level in rea, ‚ at the x 
or total readine а а ея 
ferencesthere 
treatment (the 


al difference 
f Covariance 


8. 76, Adjusted 


o 
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TABLE 5 


ANALYSIS OF COVARIANCE, PRETEST AND POST- 
TEST SCORES ON THE METROPOLITAN READINESS 
TEST, FORM R TEST 1-4, READING READINESS 


Source of Variation df SS MS F 


Groups 2 9.14 4.57 3.83 
Sex 1 1.40 1.40 1.17 
Interaction 2 1.36 0.68 0.57 


Error, within cells 227 270.78 1.19 Adjusted 
Critical Value: F, 95 (1,227) = 3, 8859 


Е 95 (2,227) = 3.0373 


TABLE 6 


ANALYSIS OF COVARIANCE, PRETEST AND POST- 
TEST SCORES ON THE METROPOLITAN READINESS 
TEST, FORMR TEST 5, NUMBER READINESS 


Source of Variation df SS MS E 
Groups 2 1.12 0.60 1.59 
Sex 1 0.90 0.90 2.55 
Interaction 2 0.24 0.12 0.35 
Error, within cells 7 79.94 0.35, Adjusted 


Critical Value: F, 95 (1,227) = 3. 8859 
F 95 (2,221) = 3.0373 


TABLE 7 


ANALYSIS OF COVARIANCE, PRETEST AND POST- 
TEST SCORES ON THE METROPOLITAN READINESS 
TEST, FORM R TESTS 1-6, TOTAL READINESS 


Source of Variation df SS MS F 
2 17.27 8.63 4.34 
ANM 1 5.82 5.82 2.92 
i 2 4.51 2.26 1.13 
Interaction 
Error, within cells 227 451.72 1.99, Adjusted 


Critical Value: E 95 (1,227) = 3. 8859 
Е 


.95 (2,227) = 3. 0373 


The major findings, basedon analysis of covariance 
were that there were significant differences between 
groups at the 5 percent level in reading readiness 
(tests 1-4) and total readiness (tests 1-6). Analysis 

of covariance revealed no differences between groups, 
significant at the 5 percent level in numbers readi- 
ness (test 5). Therefore, no further discussion of 
numbers readiness has been included, 


In order to study the magnitude of gains for each 
group, a table of unweighted cell means has been 
presented (Table 8). 


TABLE 8 


UNWEIGHTED CELL MEANS, PRETEST, POST- 
TEST, AND GAIN METROPOLITAN READINESS 
TEST, FORM R (N = 234) 


GroupA GroupB GroupC 


Test N=75 N=79  N-80 
Tests 1-4, Reading Readiness 
Pretest 41.8 38.2 40.7 
Posttest 47.4 47.8 48.8 
Саіпѕ 5.6 9.6 8.1 
Test 5, Number Readiness 
Pretest 8.2 8.1 8.4 
Posttest 11.6 12.1 12.1 
Саїпз 3.4 4.0 4.4 
Tests 1-6, Total Readiness 
Pretest 53.5 49.7 52.8 
Posttest 63.9 64.8 66.4 
Gains 10.4 15.2 13.6 


An inspection of Table 8, revealed that Group B 


d one teacher aide, experi- 
enced the greatest mean gains on tests of readi 


Scheffó's test was used to analyze 
differences for total readiness, tam Т-А a 
found that differences between Group B and ie 
A were significant at the 5 percent level 
ences between Groups C and Grou ' 
this level. Differe 


C did not approach 
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It was concluded that teacher aides can be used 
effectively to help to develop reading readiness in 
kindergarten children. Children had gained more in 
reading readiness when aides were present in the 
classroom. 


It was found that children made slightly greater 
gains in the group where only one aide workedin the 
classroom than in the group with five aides. How- 
ever,itdidnot seemto make much difference whether 
there were five aides in the room, or one. It was 
inferred that when five aides were present in the 
classroom, the teacher spent moretime training and 
supervising aides and less time working directly with 
children than she did when only oneaide was present. 
Since teachers are professionally trained to work 
with children, it was believed that directing the teach- 
er’s efforts elsewhere might not result in optimum 
pupil gain, Perhaps, if aides had been trained be- 
fore coming into the classroom to work with children to- 
ward a specific classroom objective, suchas increas- 
ing reading readiness, then aides’ classroomactivi- 
ties might not have required the degree of teacher 


planning and supervision that was necessary during 
the experiment. 


Results of this experimental study have indicated 
that the services of teacher aides can helpto develop 
reading readiness in kindergarten children. Tt is 
recommended that future research programs inves- 
tigate: 


--the kinds of training necessary to make aides 
most effective in areas where their work has 
been found to result in pupil gains, 


--the numbers of aides necessary to achieve op- 
timum pupil gains in areas where it has been 
found that aides’ services result in pupil gains. 


--other areas where the services of aides result 
in significant pupil gains. 


The researchers believe that the services of trained 
aides should free the teacher to work more closely 
with children intheareas where greatest professonal 
Skill and competence are required. Itis recommended 
that aides continue to work directly with children in 
areas where it is discovered that their services are 
most effective. However, whatever the number of 
aides present in the classroom and whatever the kinds 
of work performed, aides’ services should enable the 


teacher to spend more time withchildren rather than 
less. 


-x 
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ABSTRACT 


Democratic administrative leadership has long been hypothesized to result in high worker morale. W ithin 
the context of the elementary school, the principal is the chief administrator and the teachers occupy the major 
adult work roles. Data from 568 teachers in 52 elementary schools, classified as to administrative atmosphere, 
indicates that the principal's compliance with specific work-related norms held by the teacher is more impor- 
tant to teacher satisfaction than is the degree of democratic behavior which he exhibits. Some explanations for 


this phenomenon are suggested. 


BACKGROUND 


The human relations literature tends to divide ad- 
ministrative leadership types into two broad cate- 
gories: ‘‘democratic’’ and **autocratic." These 
categories have also been called by other names 
suchas '*member-centered"' and ‘‘leader-centered, "’ 
“non-authoritarian’’ and ‘‘authoritarian, " ‘‘non-di- 
rective’’ and “directive,” *'lenient" and *'strict. ” 
The ‘‘democratic’’ leader is usually defined opera- 
tionally as one who allows the group members some 
participation in decision making processes; who is 
available to give and to receive suggestions and ad- 
vice; and who gives praise based on standards of 
performance which are known by the group members. 
Conversely, the ‘‘autocratic’’ leader is operational- 
ly defined as one who makes decisions without con- 
sulting the group members; who gives advice but is 
unwilling to receive it; and who givesarbitrary 
praise and criticism based on standards which are 
unknown to group members. According to much of 
the research in this area, groups with ‘‘democratic” 
leaders tend to exhibit higher morale, a greater de- 
gree of job satisfaction, fewer cliques, less hostil- 
ity among members, greater approval of their lead- 
ers, and a higher degree of worker effectiveness. A 
third general type of leadership pattern has been 
designated by Lippitt and White as laissez-faire. 
The laissez-faire leader plays a passive role in so- 
cial participation and gives the group members com- 
plete freedom in decision making. Thistype of lead- 
er neither evaluates nor takes the initiative in mak- 
ing suggestions. The present study omits laissez- 
faire leadership from consideration, because the na- 
ture of most formal organizations limits the amount 
of goal-def ining which may be done by subordinates 
in the hierarchical structure, and requires evalua- 
tion of their work role performances by their admin- 
istrative superiors. It should be pointed out that the 
laissez-faire leadership type seems to betruly dem- 
ocratic in the ‘‘people’s rule” sense of the term, 
whereas what is usually designated as "democratic" 


leadership is only democratic relative to *autocrat- 
ic" leadership, and may be more precisely labeled 
“lenient, `` **participant-oriented," etc. Regardless 
of the label, according to most of the human rela- 
tions studies, ‘‘democratic’’ adminstration seems to 
constitute a very effective type of leadership behav- 
ior. 


Although the human relations modeltends to be a 
bureaucratic one, ^ there are, of course, many dif- 
ferent types of bureaucratic organizations. Etzioni 
categorizes formal organizations in ter ms ol the 
major types of power used. Utilitarian or ganiza- 
tions are those which utilize remunerative power or 
‘‘control over material resources and rewards "id 
Normative organizations use normative powerorthe 
“allocation and manipulation of symbolicrewards 
and deprivations.'" Coercive organizations use co- 
ercive power or the threat or actual application of 
physical sanctions. An example of a utilitarian or- 
ganization is an industry; of a coercive Organiza- 
tion, a prison; of a normative organization 


r ‚ a hos- 
pital. $ Most human relations studies have been con- 
cerned with utilitarian organizations, Even where 

the organizations hav 


: € been normative ones 
pervisor-subordinate relat: 


tended to be those in which 
ation play a dominant role. 


1 п ‚ the su- 
ionships studied have 
Foercion or re muner- 


s on the relationship be- 
ive roles – one 
= ina 
dae organization. The roles 
and principal. The organization is the i 
mentary School. The Study examines i a =з 
ity of current human relations theory to tie aor 
as a normative, bureaucratic institution M 


adminis- 
predo minantly nor- 
are those of teacher 


Teachers, besides bein: 
organizations, differ in ot 
dustrial workers who hav 
human relations rese 


& members of normative 
her Tespects fram the in- 
: € been the Subjects of most 
arch, They tend » at the 
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elementary school level at least, to be women rath- 
er than men. As women, given the valuesof our so- 
ciety, they may be expected to plan theircareerson 
the bases of factors extrinsic to the work situation. 
These factors may included such life circumstances 
as marriage or spinsterhood, wealth or poverty, 


norms. It has been shown in these same studies 
that teachers hold clear norms concerning what con- 
stitutes acceptable behavior by the principal. The 
most important norms which teachers hold concern- 
ing the principal’s behavior are: 

1. He should “аск up” (i.e. , support) the teach- 


children or absence of children. ? Teachers also 
tend to have had more education than most of the 
assembly line workers studied. Their jobs are 
less routine than the assembly line job. The teach- 2 
er has more decisions to make and is less closely 
supervised than the industrial worker. For all 
these reasons: Sex, education, autonomy ,and lack 
of routine, the possibility exists that the teache r 
may respond to certain types of administrative b e- 
havior in a somewhat different mannerthan would 
an industrial worker. Still, the teacher is a work- 
er in a formal organization, andher work is subject 
to orders and evaluation from the principal who is 
her hierarchical Superior. She should then, if the 
human relations studies are Correct, respond to 
certain general adminstrative practices inthe same 
direction, if not with the same intensity as industri- 
alworkers. The major questions with which this 
paper is concerned are the effects of particular 
types of adminstrative atmosphere on the teacher's 
evaluation of the principal's leadership ability; on 
her work satisfaction; on the amount of cohesiveness 
the teachers believe exists among themselves; on 
the perception of the existence of hiera rchical 
cliques; on the attitudes of the teachers toward the 
pupils and their parents; and on the teacher’s feel- 
ings of freedom to consult the principal. These are 
the variables which in most human relations studies 
are believed to be dependent on the degree of ‘‘dem- 
ocratic’’ or *'autocratic"' behavior exhibited by the 
leader or supervisor of the work group. 


er in front of parents, even when he considers 
the teacher to be “wrong.” If he criticizes 
her, it should be done privately. 

+ He should “Баск up" the teacher in f ront of 
pupils. Again, if the principal believes that 
the student is “right, "" the teacher should be 
told this privately. She shouldnever be criti- 
cized in the presence of the pupil. 

3. He should be a good disciplinarian. 10 
4. He should exhibit democratic behavior. 

Although these are all norms with which the teacher 

expects the principal to comply, we would expect if 

‘‘democratic’’ behavior is primarily useful as a 

means to the knowledge of subordinate nor ms , ad- 

ministrative compliance with norms 1 through 3 to 

be more important in determining aspects of teacher 
satisfaction than compliance with norm 4. Asaform 
of shorthand throughout this paper, the norms of 

“backing” and disciplinarianship are referred to as 

“ends norms; administrative compliance with the 

ends norms is called "positive normative compli- 

ance;’’ non-compliance withthe ends norms is called 

"'negative normative compliance." Theadminis- 

trator (the principal in this case)can exhibit posi- 

tive compliance with the ends norms andyet be ‘‘au- 
tocratic, " or he can exhibit negative compliance 
with the ends norms and be “democratic, ” or he can 
comply or refuse to comply with both sets of norms. 

This combination of possibilities yields a typology of 

four kinds of administrative atmosphere: democrat- 

іс- positive (compliance with ends norms), autocrat- 


ic- positive, democratic- negative, and autocratic- 


HYPOTHESIS negative. 


Merton suggests that the effective supervisor 
must know the norms of his subordinates. Other- 
wise he may unintentionally violate these nor ms 4 
and the violations may cause resentment which even- 
tually leads to the undermining of the leader 's au- 
thority and to the inefficient pursuit of organization- 
al goals.’ Since subordinate participation and effec- 
tive two way communication have beenthe main var- 
iables used to operationally define “democratic”’ 
leadership, it seems quite possible that from the 


administrator’s position, ‘‘democratic’”’ leadership 
functions as a 


Subordinate nor ms. 


The basic hypothesis of this study is that for any 
index of teacher satisfaction traditionally thought to 
be positively influenced by “democratic” leadership, 
a stronger positive influence will be exerted by per- 
ception of the principal's positive compliance with 
the ends norms. The highest percentage of most 
satisfied teachers shouldbe in “democratic’’ schools 
where the principal shows positive compliance with 
the ends norms. The next highest percentage of 
most satisfied teachers should be found in “‘autocrat- 
ic” schools in which the principal exhibits positive 
normative compliance. The percentage of most sat- 
isfied teachers on any given index of satisfaction 
should decline further in “de mocratic" schools 
where the principal is non-compliant and should 
reach its low point in autocratic schools with non- 
compliant principals. Table 1 provides a graphic 
illustration of this hypothesis, as wellasan example 
of the type of percentage table by means of which the 
hypothesis is examined. 

н nt, is more i in- 
istrative adherence to “democratic” па папай 
nine boat бы behavior is a means to 
the subordinates hold. important norms which 


THE SAMPLE 


The data for this study are obtained froman anal- 
ysis of the questionnaire responses of 1, 250 white 
teachers in 108 North Carolina elementary schools, 11 
All of the subjects come from schools 
to 25 teachers, the modal size (9 


Tt ha; i 

Me most oh арап in Several studies that teachers 
jobs when the principal 

ment on occupational 


ied with their 


and the teachers are in agree’ 
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autocracy of administrative atmosphere (see next 
section for further explanation), teachers from 
Schools with responses rates of under 50 percent 
are omitted from the sample. Thisrestriction, the 
restriction on size of school, andanother one which 
eliminates from the sample teachers who are also 
part of administration, lowers the sample number 
from the 1, 586 (49. 3 percent of the white elementary 
school teachers in the State in the spring of 1962, 
when the data were collected) teachers who answered 
the questionnaire to 1, 250 whose questionnaires are 
usable. Ninety-four and three-tenths percent of the 
sample is female. Most of the teachers have more 
than 30 pupils in a classroom. Over half of the 
teachers are over forty years $t age; about one quar- 
ter of them are under thirty. 1 


MEASUREMENT 


In order to test the hypothesis, each of the 108 
elementary schools in the sample (it willbe recalled 
that each of the schools has a response rate of over 
50 percent) is classified according to autocracy of 
administrative atmosphere. The classifications are 
based on the response consensus of the teachers in 
each school to a six item Likert scale concerning 
the behavior of the principal. Each teacher has а 
score which is based on her responses to the scale 
items. The mean score is computed from the indi- 
vidual scores of the responding teachers in each 
school, thus giving each school an administrative 
atmosphere score, based onthe collective responses 
of the teachers, The schools are grouped 
into three categories: low autocratic or **democrat- 
їс?” schools (314 teachers), noconsensus (682 teach- 
ers) and highly autocratic Schools (254 teacher s). 
For the purposes of examining the hypothesis, only 
the schools in which there is consensus concerning 
the autocracy or lack of autocracy of the principal 
are included in the Study. This additional restric- 
tion reduces the number of schools analyzed f rom 
108 to 52 and the number of teachers from 1, 250 to 
568. The Likert items on which the teachers were 
asked to rate their principals are as follows: 

1. Allowing teachers freedom on the job 

l.... Outstanding 

2.... Above Average 

3.... Average 

4.... Below Average 
2. Letting teachers participate in decisions that 

concern them (same categories as 1, above) 

3. Identifying himself and his interests with the 
teacher group (same categories as 1, above) 

4. Supporting and encouraging teachers in new ways 
of doing things (same categories as 1, above). 

5. Does your principal get together or talk informal- 
ly with teachers or does he confine himself strict- 
ly to business? 


1.... Often associates informally with teachers 

2.... Associates informally with teachers from 
time to time 

3.... Rarely associates informally with the 
teachers 

4.... Never associates informally with the 
teachers 


6. How often does the principal ask you for advice 
or ion your opinion on school matters affecting 
you? 

0.... Less than once a month 
1.... 1 to 3 times a month 


- 4to 6 times a month 

- Tto 9 times a month 

- 10 times a month or more 

At test was used to examine the discriminatory 
power of each item. It was found that each item dis- 
criminated between the high and low quartile of au- 
tocracy, scores at a probability level of less than 
. 001. A high score on the scale indicates a per- 
ception of a highly autocratic principal. 


2 
3. 
4 


A simple index of normative compliance (compli- 
ance with the ends norms) was constructed from the 
following three items: 

1, Can the principal be counted on to stand upfor 


the teacher when a parent comes in with a com- 
plaint? 


l.... Can always be counted on 
2.... Can usually be counted on 
3.... Can sometimes be counted on 
4.... Can hardly ever be counted on 
9.... Can never be counted on 


2. Can the principal be counted on to stand up for 
the teacher when a student comes in with а com- 
plaint? 

l.... Can always be counted оп 

2.... Can usually be counted on 

3.... Can sometimes be counted on 

4 - Can hardly ever be counted on 

5.... Can never be counted on 

I 

1 

2 

3 

4. 


5 your principal a good disciplinarian? 
. Outstanding 
- Above Average 
. Average 

+++. Below Average 
The items are scored according to their code values, 
In items 1 and 2, codes 4 and 5 are comb ined be- 
cause of small response numbers into one category, 
‘t4. ” Scores of 3, 4, or 5 are called “positive nor- 
mative compliance." Scores of бог angy e are 
called ‘‘negative normative compliance, ’’ 


FINDINGS 


According to the literature, many varied 
of the work situation, especially those as 
ly pertaining to work Satisfaction 


aspects 
pects close- 
; tend to be influ- 
ship behavior, On 


will be 


ample, 
re not used, 


Human relations 
жү relationshi 
Ship and subordinate evaluati 
leadership ability. mi iei m 
Were asked to rate the ** 
of their principal. 


р between 


42 THE JOURNAL OF EXPERIMENTAL EDUCATION 
TABLE 1 


PREDICTED PERCENTAGES OF TEACHERS EXHIBITING HIGHEST SCORES ON ANY INDEX OF 
SATISFACTION RELATED TO THE WORK SITUATION 


Administrative Atmosphere 


Normative Compliance Democratic Autocratic 


Positive highest percentage next highest percentage 
Negative next lowest percentage lowest percentage 
TABLE 2 


PERCENTAGE OF TEACHERS ACCORDING HIGHEST LEADERSHIP EVALUATION TO THE PRINCIPAL 
(*OUTSTANDING'' RESPONSE) 


Democratic School Autocratic School 


Normative Compliance Percent Na Percent ма Total 
Positive 49.6 252 19.5 128 380 
Negative 9.4 53 0.0 117 170 


N? - the number of teachers who answered the questions in e 
percentages were computed. For example, 49.6 perce 
(democratic schools where the principal complies with 
of outstanding leadership ability. None of the 117 te. 
ing” leadership rating to their principal. 


ach type of school - the bases from which the 

nt of the 252 teachers in democratic- positive schools 
the “ends” norms) grant their principal an evaluation 
achers in autocratic-negative schools gave an **outstand- 


which they should be read. 


of the principal's positive compliance with the ends 


horms than to the autocracy or democracy of his be- 
havior. 


“Democratic” 


leadership has been related to 
Work satisfaction, 


An index of work satisfaction was 

constructed from two items which asked the teacher 

about her Satisfaction with her Specific job and with 
her teaching career. The items are: 

1. How Satisfied are you with your present teaching 
position, compared with what you imagine most 
teaching positions are like? 

1.... It is the best teachin, 
istically imagine. 
2.... Itis better than mo: 
3.... Itis about as 
Д tions, 
:--- It is inferior to most teachin iti 

2. Speaking for УНШ as eris. 

hé heel an aS how Satisfying has teach- 


& position I can real- 


St teaching positions, 
good as most teaching posi- 


l.... The most Satisfyi 
isfying career І сап imagine 
am È e Satisfying than most careers А 
+++. About as Satisfying as most car 
4.... Less Satisf rere. 


DE 


All of the five tables which follow are alike in regard to the 


way in 


Scores on the index can run from 2 to 8inthe direc- 
lion of most to least favorable. Highest work satis- 
faction (a score of 2) is more closely related to the 
perception of the principal's compliance with the 


ends norms than to his “democratic” behavior, (See 
Table 3.) 


People who work in organizations in which there 
is “democratic” leadership have been shown by 
much human relations research to have higher mo- 
rale. Because the teacher works so closely with 
the organizations's “clients, " her opinion of the 
“clients’’ should be one index of her morale. Teach- 
ers were asked to “rate the attitude of the students 
in your school” and to “rate the attitude of the par- 
ents. '' 15 The four response alternatives range from 
outstanding to below average, a scoreof 2 indicating 
the highest rating on this two item index. Again the 
tendency appears for highest rating of clients to re- 
late more strongly to the positive compliance of the 
principal with the ends norms than to his ‘‘demo- 
cratic" behavior, (See Table 4.) 


An index measuring feelings of cohesiveness 


ü 


TABLE 3 
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PERCENTAGE OF TEACHERS RECEIVING HIGHEST SCORE ON WORK SATISFACTION INDEX 


(SCORE OF 2) 


Democratic School 


i a 
Normative Compliance Percent N 


Autocratic School 


Percent ма Total 
Positive 25.1 247 11.1 126 pra 
Negative 9.4 53 4.4 114 


N? = the number of teachers answering the questions in each type of school - the bases from which the 


percentages were computed. 


TABLE 4 


PERCENTAGE OF TEACHERS GIVING HIGHEST RATING TO “SCHOOL CLIENTS" 


Democratic Schools 


Autocratic Schools 


Normative Compliance Percent Na Percent ма Total 
Positive 10.5 248 4.0 126 374 
Negative 3.8 52 1T 117 169 


N® = the number of teachers answering the 
percentages were computed. 


with other teachers was constructed from two items 
which ask the respondent how well she likes the oth- 
er teachers in the school and how similar she feels 
tothem. The questions are: 
1. How well do you like most of the other teachers 
in your school? 
1.... Very Well 
2.... Fairly Well 
3.... Not Much 
4.... Not at All 
2. How similar are you to other teachers in your 
school in your values and outlook on life? 
1.... Similar to Most 
. Similar to About Half 
3.... Similar to Fewer Than Half 
The index scores can run from 2 to 7inthe direction 
of most to least favorable. The highest cohesive- 
ness score is a score of 2. Table 5 shows that the 
hypothesis that cohesiveness has a greater relation- 
Ship to compliance with ends norms than to **demo- 
Cratic" behavior is upheld in the case of positive 


compliance but is not supported in the case of non- 
compliance, 


Clique formation has been related to autocratic 
leadership in some of the human relations studies. 


questions in each type of school - the bases from which the 


An index to measure perception of this w 


ted from three items which ask about th 
of influential cliques 
questions are; 


1. In some schools i 
Я anythin: 
to say about the running of the school, on ly ihe. 
er circle count for m uch. 


as construc- 


e existence 
of teachers and parents, The 


2.... Dominated by an inner circle of teachers 
l.... Most teachers have an equal voice 

2. Does a teacher or sroup of teachers whi has 
(have) been in the School a long time have Wee 
pecially Strong voice in influen Cing th ES ей 
the principal deals with teachers, studente wes 
ma ar other important School matters? r 
l.... Yes 

3. 


Do socially promin 
in the school have 
influencing the wa 


ent people who h a h 
ave ге 
А : Children 


cially strong 


: voice iy 
y the princi ' 

= Pal deals 
achers, Students, curriculum, ор ther м. 
tant school Matters? У oer роке 


. No 
l.... Yes 
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TABLE 5 


PERCENTAGE OF TEACHERS PERCEIVING HIGHEST COHESIVENESS WITH OTHER TEACHERS 


Democratic Schools 


Normative Compliance Percent ма 


Autocratic Schools 


a 


Percent N Total 
Positive 12.5 247 66.9 ет УШ 
Negative 50.9 53 54.2 11 


N? = the number of teachers answering the questions in each type of school - the bases from which the 


percentages were computed. 


TABLE 6 


PERCENTAGE OF TEACHERS CLAIMING NO CLIQUES EXIST IN THEIR SCHOOL 


Democratic Schools 


Autocratic Schools 


Se аана 

Normative Compliance Percent Na Percent N Total 
Positive 78.9 246 66.1 124 370 
Negative 41.1 51 37.8 111 162 


N? = the number of teachers answering the questions in each type of school ~ the bases from which the 


percentages were computed. 


Index scores can run from 1 to 4 in the direction of 
most to least favorable. Absence of cliques tends 
to have a greater relationshipto the principal'scom- 
pliance with the ends norms than to his ‘‘democrat- 
ic" behavior. (See Table 6.) 


The subordinate's subjective feelings of freedom 
to consult the supervisor should relate very closely 
to his “democratic” behavior, according to human 
theory, since if à Supervisor is ''democratic," he 
attempts to establish effective two way communica- 
tion. It will be recalled that the Likert scale, which 
is used to categorize schools as to administrative at- 
mosphere, asks about the behavior of the principal: 
how often he asks the teacher for advice, how often 


SSociates informally with the teacher, etc. I 
Order to establish the teacher’ j : lings 


definition) th 

l. When you 
sult with 
loa 


not a part of its 
е teacher was asked: í 

have problems, do you feel free to con- 
your principal about them? 
eel very free to consi 


AME f ult with principal 
| in" little hesitant to consult with prin- 
+++. Feel ve i i i 
M ry hesitant to consult with princi- 


ER. 


The percentage of teachers feeling ‘‘very free to con- 
sult with principal" is more closely related to the 
compliance of the principal with the ends norms than 
to his “democratic” behavior. (See Table 7.) 


SUMMARY AND DISCUSSION 


It is hypothesized that the compliance of the ele- 
mentary school principal with what have been called 
the “ends norms” of the teacher has a stronger re- 
lation to various indices of teacher satisfaction than 
does his “democratic” behavior, Thereisatenden- 
cy for the hypothesis to be upheld in all but one of 
the six relationships that were examined inthis 
Study. The positive compliance of the principalwith 
the “ends norms” (*backing" the teacher and dis- 
ciplinarianship) has a greater influence than does his 
"democratic" behavior on the teacher's high evalu- 
ation of the principal's leadership, her work satis- 
faction, her high evaluation of school *'clients, "her 
perception of the absence of cliques, and her feel- 
ings of freedom to consult the principal. The same 
trend seems to exist to some degree in the case of 
feelings of cohesiveness with other teachers in the 
School. However, teachers in “autocratic”? schools 
who perceive their principals to be no n-compliant 
with the ends norms feel a slightly greater degree 


pe 
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TABLE 7 


PERCENTAGE OF TEACHERS FEELING VERY FREE TO CONSULT PRINCIPAL 


Democratic Schools 


Autocratic Schools 


Normative Compliance Percent ма Регсепі Na Total 
Positive 96.8 251 18.0 127 378 
Negative 71.1 53 42.7 117 170 


N? = the number of teachers answering the question in each type of school — the bases from which the 


percentages were computed, 


of cohesiveness with other teachers than do teachers 
in ‘‘democratic’’ schools who perceive their princi- 
pals to be non-compliant. The findings of this study 
give no explanation for this tendency, but a guess 
may be hazarded that the teachers in the "'autocrat- 
ic” schools where the principals are non-compliant 
are so miserable that they band together for self 
protection and comfort. The teachers in the ‘‘demo- 
cratic” schools may have at least some tw o- way 
communication outlets for cammunicating their dis- 
content to the principal and perhapseven some hope 
for having something done about it. 


The question arises as to why the ‘“‘ends norms” 
of “backing” and disciplinarianship are more im- 
portant to the teachers than the norm which requires 
the principal to be **democratic'" (hypothesized to be 
a ''means' norm). Every organization may have 
its own ends norms, and their existence and nature 
should be discovered in further studies; but why are 
these particular norms so important to the teacher ? 
The findings of previous research suggest some an- 
swers to this question. 16 For one thing, the teach- 
er is a professional or at least (perhaps more ac- 
curately) a semi-professional. She is trained touse 
normative control over her students. However, her 
Students are very young and often are not highly mo- 
tivated. They may have to be coerced into following 
the organizational rules “ог their own good, " The 
more time the teacher must spend in coercion the 
less time she has to teach. If the principal cannot 
be counted on for help with disciplinary problems - 
if, for example, he frequently sides withthe student, 
the teacher's classroom authority is weake ned. 
When this happens, the students are quick to realize 
it, and the disciplinary problems become mor e 
widespread, more frequent, and more difficult to 
control. The teacher who is caught inthisdilemma 
cannot teach, which is what she is trainedtodo. She 
becomes at best a baby sitter and at worst a jailer. 
She cannot trust her principal for help, because he 
may side with the next unruly child or its parents, 
thus increasing her problems. She then becomes 
dissatisfied with her job. On the other hand, if the 
teacher can maintain her authority in the classroom, 
she Can get her job done. The behavior of the prin- 
cipal with regard to the ends norms toagreatextent 
determines whether the teacher's classroom author- 
ity will be upheld. The norms which demand that a 


principal ‘‘back’’ a teacher in front of parents and 

pupils and that he be a good disciplinarian,are 
norms which protect the teacher from those with 
whom she is in constant contact — the children. She 
is very visible to them and there isa constant threat 
to her authority unless the principal complies with 

the **ends norms. ” 


The norms which demand that the principal ex- 
hibit democratic behavior protect the teacher's po- 
sition in the internal organizational structure with- 
out immediate regard to the clients (parents and pu- 
pils). These norms protect the teacher from the 
arbitrary use of power by the principal and insure @ 
her rights to make her views known. However, be- 
cause the teacher is not constantly visible to the 
principal as she is to her students, heis not the 
threat to her authority that they are, unless his be- 
havior in some way negatively influences the chil- 
dren’s concept of her. If the principal complies 
with the teacher’s views regarding the ends norms, 
she has very little need to communicate with him at 
all and thus little need for “democratic” behavior 
on his part. (Although, if the principal’s behavior 
complies with the ends norms, the teacher may like 
and trust him so much that she will co 
with him freely. ) “Democratic”? behav 
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FOOTNOTES 


1. Ronald Lippitt and Ralph К. White, “Ап Experi- 


mental Study of Leadership and Group Life, ” 
Readings in Social Psychology, Editors, Guy 
E. Swanson, Theodore M. Newcomb, and Eu- 
gene L. Hartley (New York: Henry Holt and 
Company, 1952), pp. 340-355. For some oth- 
er discussions of leadership effects see:Peter 
M. Blau, Bureaucracy in Modern Soc iety 
(New York: Random House, 1956); Peter M. 
Blau and W. Richard Scott, Formal Organiza- 
tions (San Francisco: Chandler Publishing 
Company, 1962). Blau and Scott have an in- 
teresting four-fold classification of formal or- 
ganizations. The organizations are classified 
in terms of their prime beneficiary. Accord- 
ing to this scheme, the only typeof formal or- 
ganization which necessitates a democratic in- 
ternal structure is what they call ** mutual- 
benefit associations, ’’ where the prime bene- 
ficiary is the membership of the organization 
(p. 43), See also Cecil A. Gibb, ‘‘ Leader- 
ship, ’? Handbook of Social Psychology, Volume 
II: Special Fields and Applications, Editor, 
Gardner Lindzey (Cambridge, Mass.: Addi- 
Son-Wesley Publishing Company, 1954), pp. 
877-917; Kjell Harnvist, Adjustment, Leader- 
ship and Group Relations in a Military Train- 
ing Situation (Stockholm: Almqvist and Wik- 
sell, 1956), pp. 120-122; Louis Kriesberg, 
‘Industrial Sociology, ’’ Sociology inthe United 
States of America, Editor, Hans Zetterberg 
(New York: UNESCO, 1956), pp. 71-77; Mel- 
vin Seeman, Social Status and Leadership: The 
Case of the School Executive, Research Mono- 
graph No. 35 (Columbus: Bureau of Educa- 
tional Research and Service, Ohio State Uni- 
versity, 1960); Hanan C. Selvin, The Effects 


of Leadership (Glencoe, Ш. : The Free Press, 
1960). 


2. For a discussion of assumptions underlying the 


3. Amitai Etzioni, A Comparative An 


4. Most human relations s 


use of the human relations model see: Robert 
Dubin, ‘Psyche, Sensitivity, and Social struc- 
ture, ” Leadership and Organization, Robert 
Tannenbaum, Irving R. Weschler, and Fred 
Massarik, Editors (New York: McGraw-Hill 
Book Company, Inc., 1961), p. 403. 

alysis of Com- 
plex Organizations: On Power, Involvement 
and Their Correlates (New York: Free Press 


01 Glencoe, Inc., 1961), all definitions on page 
ive. 


tudies which have focused 
xample, have been c on- 
neipal- student relationship 
dent relationship, both of 


on the school, for e: 


teachers ag 


vard Educati, 


o —xcational Review, 


E — 


10. 


11, 


12. Twenty-five and еі 


13. A 


29 (Fall, 1959), p. 373. Twelve hundred out 
of a total of 1,250 white elementary school 
teachers in the present study answered a ques- 
tion about the way they felt regarding their fu- 
ture as teachers. Forty-two and three-tenths 
percent of them wanted to go into some other 
kind of work, including becoming full-time 
housewives. 


- The Bachelor of Arts or Science degrees are held 


by 73. 5 percent of the teachers in the sample. 
Most of the others (214 or 17. 5 percent of the 
total) have done some graduate work. 


. Refer to sources in footnote 1. Although human 


relations studies relate **democratic" leader- 
Ship to worker effectiveness, this paper does 
not deal with this aspect. It is very difficult 
to construct a valid measure of teacher effec- 
tiveness, especially on the elementary and ju- 
nior high school levels. 


Robert К. Merton, Social Theory and Social 
Structure (New York: Free Press, 1957), pp. 
339-340. 

See Peter M. Blau and W. Richard Scott, op. cit. 
р. 143 for a discussion of the processes by 
which lenient supervision may result in an ex- 
tension of the legitimation of the supervisor’s 
authority. 

Howard S. Becker, ‘Тһе Teacher inthe Author- 
ity System of the Public School, ” Complex Or- 
ganizations: A Sociological Reader, Editor, 
Amitai Etzioni (New York: Holt, Rinehart, and 
Winston, Inc., 1961), pp. 243-251. Robert Nel- 
son Bush, The Teacher -Pupil Relationship 
(New York: Prentice-Hall, 1954). Donald C. 
Moyer, ‘‘Teacher's Attitudes Toward Leader- 
Ship as They Relate to Teacher Satisfaction, '' 
(Unpublished Master's Thesis, University of 
Chicago, 1951). Miriam Wagenschein, **Real- 
ity Shock, '" (Unpublished Master's Th 
University of Chicago, 1951). 

In the spring of 1962, when the questionnaires 
were mailed, North Carolina schools had 
Started to become racially integrated. How- 
ever, in most cases, the schools were still 
distinguishable by racial categories, This pa- 
per deals only withteachers from white 
Schools since race introduces many new com- 
plications into the already complex relation- 
Ship between administrative atmosphere and 
aspects of job satisfaction. 

ght-tenths percentof the teach- 

ers (314) are 29 or younger; 16. 2 percent are 
between 30 and 39 (197); 23. 0 percent are be- 
tween 40 and 49 (280); 27, 4 percent (333) be- 

tween 50 and 59; and 7.6 percent (92) are 60 

years old or older. The total number of re- 

spondents to this question is 1216. 

П but item 6 in this scale are selectively drawn 

from Melvin Seeman's interesting monograph, 

Social Status and Leadership: The Case of the 

School Executive, Research Monograph No, 35 

(Columbus: Bureau of Educational Research 

and Service, Ohio State University, 1960). The 

Question may be raised as to why the teacher's 

Perceptions of the principal’s behavior we re 

obtained, rather than the principal's ser -ге- 

Ports of their actual behavior. An assumption 

is being made here that the Perceptions an indi- 

vidual has about a situation are more important 

in shaping his attitu the Situation 


esis, 


ides Concerning 
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than are the **objective facts. " Further, since believed that a potential ''average'' rating on 
all of the respondents from each school rate more than one item would be too low a rating 
the principal on the same items, the average for the positive category. Therefore, a score 
of the collective responses should serve as a of 5 or under is placed in the positive cate- 
basis of inference of actual behavior. The self- gory and a score of 6 or over is placed in the 
report of the principal may be honest in that negative category. Because the literature 
he is reporting what he thinks he does, but his stresses the teachers' wishes for principals 
actions may not be perceived or reacted to in who ‘Баск them up’’ and are good disciplinar- 
the way in which he intended. ians, and because the literature does not give 
]4. A score of 3 means perception of the highest special importance to any one of these behav- 
principal compliance on all three items; а ioral aspects, the items are given equal weight. 


score of 4 means perception of highest princi- 
pal compliance on two items and above aver- 
age compliance on one; a score of 5 means 
either perception of highest compliance on one 


15. This wording was a result of a pretest in which 
most complaining teachers said such things 
as **1 don't like their attitude," 


item, and above average on two, or average 16. See especially, Becker, op.cit. ; Bush, Op. cit. ; 
on one and outstanding on two. The cutting Etzioni, A Comparative Analysis, op cit.; 


point is drawn at a score of 5, because it is and Wagenschein, op. cit.; and Moyer, op. cit. 
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THE BORDERLINE STUDENT AS A MEMBER OF 
A SEPARATE SCHOOL GROUP 


DONALD R. HETZNER 


Medina School System 
New York 


ABSTRACT 


Although the Medina, New York, Prevocational Program for ''slow learners” had been in operation for 11 
years there was no adequate proof that the Prevocational section varied significantly, on the basis of mean in- 
telligence, from either the Special Education or academic section of the school system. Intelligence quotient 
Scores were gathered and mean intelligence, standard deviation, and confidence interval were established for 
each of the sections. Null hypotheses were then used to test for significant differences between the centraltend- 


encies. This established that Prevocational, Special Education, and the academic section were discrete groups. 


Educational facilities drawing their enrollment from a representative section of the general population will, 
under Medina criteria, have approximately 17-20 percent “510% learners” in their school population, If the 
needs of these borderline students are to be fulfilled it is the responsibility of the schools to design curricula 
tailored to a group falling midway in a progression between Special Education and academic classes. 


SINCE 1955 the Medina, New York, School Sys- 
tem has offered a program for teenagers whose 


ability to achieve the standards of academic courses 
is doubtful. For 7 years the program was offered 
only in the seventh and eighth grades; in 1961 it was 


expanded to include the ninth grade of the Senior High 
School. 


After 1962 the Prevocational Program was 
expanded a year at a time until, by 1965, it covered 
grades seven through twelve and graduated its first 
class. The original program, and its expansion to 
include junior and senior high school, was basedon 
the idea that the maintenance of a comprehensive com- 
munity high school dictated that the needs of all sig- 
nificant groups of s Thus, those stu- 
fente who are, “academic slow 

» 
mis paa tailored to their 
hem in school 


MENTAL ABILITY AND READING DEFICIENCY 


The Medina Pre 
educati 1: 


whom have intelligence quotients inthe range 75-90, 
are considered ‘‘slow’’ only in regard to academic 
capacity; in other areas of endeavor they may rate 
average or better, Of the three factors mentioned 
mental ability and reading level, as measured by 
standardized tests, are considered the most unbiased 
estimators used to evaluate and place students, par- 
ticularly as all past intelligence and reading tests 


are taken into account when considering a student 
for the program. ! 


At present, of the 135 students ingrades 7- 12 with 
intelligence quotients in the range 75- 90, ninety-eight 
are in the Prevocational Program. Of the 168 stu- 
dents participating in the plan only one student was 
reading at grade level when admitted. From all in- 
dications of a study now in progressthere is a posi- 
tive though non-linear relationship between below- 
normal intelligence and reading deficiency.? The 
remaining 28 percent of the students in the program 
have a mean intelligence quotient of 95. 83, above the 
cut-off point of 90 but placed in the program because 
of what are considered irreversible readin g de- 


When dealing with students having intel- 
1 ; g inte 
ligence quotients in the range 91-100 the Prevocational 


Department has found that il i 
mre pupils reading more than 


w grade levelare so severely hindered 
t any success in an a i dos 
cademic program i 
doubtful. There are sever, ue verifiabl 


al empirically verifiable 


conclusions that can be drawn from the existing data: 


às 
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TABLE 1 


ACHIEVEMENT TESTS (Median Number oi Months Below Grade Norm) 


Reading Arithmetic Composite Age Difference In 
Iowa Iowa Iowa Months From Median 
Basic Reading Basic Basic Age of Rest of 
Grade Skills Stanford Skills Skills Grade 

7 -19 -16 -8 -14 +12 

8 -14 -19 -10 -14 +14 

9 -12 -18 -6 -10 +13 

10 -14 -18 -10 -12 +11 

ll -14 -17 -10 -12 +11 

12 -15 -17 -8 -12 +15 


From ‘‘The Prevocational Program in Grades 7-12 of Medina Central School, Sixth Progress Report” (Medina, 
New York: Medina Central School, June, 1967) ( Mimeographed ). 


a student of normal intelligence can function in an 
academically oriented class even witha severe read- 
ing deficiency, but itis impossible for a student with 
limited intelligence and a reading disability to com- 
pete in an academic class, Readingability is ex- 
tremely critical for students who are on the border- 
line of entering the program by the results of their 
Scores on intelligence tests. These are pupils who 
have intelligence quotients in the range, roughly, of 
91-100 and severe reading problems. There is some 
Question within the program regarding what is tobe 
considered a reading problem serious enoughto im- 
pair any chance of success in anacademic program. 
The usual practice is to consider a student with an 
intelligence quotient of over 90forthe programonly 
if he is a year or more behind in reading, has failed 
a grade and perhaps is in danger of failing again. 
By following these entrance requirements close ly 
students are placed in the program because of limited 
mental ability and an irremedial lack of reading 
skills, not because they are *'underachievers''(4), 
Table 1 presents an overall view of the achievement 
levels and age differences between the Prevocation- 
al classes and the grade norms. 


THE NEED TO VERIFY THE REASON FOR THE 
PLAN’S EXISTENCE 


Though the Medina program for ‘‘slow learners” 
has existed for 11 years, there has never been an 
adequate statistical proof that the Prevocationaland 
Academic sections are significantly different, This 
also seems to be the case in other schools where 
academically slow students are grouped separately. 
It does not suffice as a proof merely to state that 
Students in a slow group have intelligence quotients 
between 75-90 inclusive, or that the mean intelligence 
of the group is say, 87. Inthe first case the com- 
position of groups will vary from year toyear and/ 
or group to group, and the intelligence curve or the 
Sample may be skewed toward the lower or, more 
likely, the upper boundary of the range. A mean 


intelligence quotient in isolation, as in the second 
case, is valueless unless used for prediction or in 
valid comparison with other groups. Thus it is nec- 
essary to establish a confidence intervalfor the mean 
intelligence of the ‘slow learner" groupanda statis- 
tical measure, in this case mean intelligence of all 
Prevocational students in Medina, whichcanbe used 
to test hypotheses about the ‘‘slow learner" group. 


THE ERROR OF OBSERVATION 


The original impetus behind the Prevocat ion- 
al Program came from teachers who recognized the 
inability of certain students to Cope with an academic 
program--still--they were pupils who definitely did 
not belong in Special Class. Because ac ademic 
Courses were often geared toward college entrance 
it was felt that such study did not meet the needs of 
students whose last aquaintance with formal educa- 
tion would be, at most, high school. The Solution 
was to place these students in a program suited to 
their abilities, a program where the formation of 


would seem to indicate nothing mor 
normal teen-agers. In fact, one wou 
because of the stress placed onacc 


attitudes, is less given to fadish d; 
modes than other groups. 


this isa group different fro 
demic classes. The truth is thay zh thee OL poor aca- 
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level reading, produces to view an outwardly aver- 
age looking group of teenagers. But, concealed un- 
der overt appearances are individuals who, unless 
guided toward social responsibility, become tomor- 
row's unemployables. Extrapolating from the data 
at hand--these are individuals who, in most school 
systems, are not recognized as being members of 


à valid group and do not have their life needs fulfilled. 


In the following sections null hypotheses will beused 
to test for significant differences in mean mental 


ability between the Special Education, Prevocational, 


and Regent's sections of the Medina school system. 
“Slow learners" form a group that, though over- 
looked, still exists and whose needs cannot be met 
by having its members scattered in academic groups 
and subject merely to individual counseling. They 
are not isolated cases--they are members of a val- 
id group that exists in every school that draws its 
students from a normally distributed population and 
Should be treated accordingly (2). 


PURPOSE OF THE STUDY 


Students in the Prevocational Plan are selected 
and placed by criteria evolved through long term 
observation and research. Selection procedure, 
however, is not the point of this paper. Rather, 
the center of interest is the question: Are Prevoca- 
tional students, once selected and placed in a homo- 
geneous group by arationaland coherent set of stand- 
ards, the nucleus of a group significantly different 
in mental ability from the Special Educationandaca- 
demic sections of the school system (4). It is realized 
that the make-up of a groupis the function of the se- 
lection criteria. In this case, however, there isa 
group already in existence, Therefore, the fallacy 
of positing a group, setting the selection criteria, 
and then testing the group is avoided. Althoughthe 
composition of a group will vary directly in relation 
to the selection procedure, if the selection process 
is held constant, statistical technique can be applied 
to determine whether a Prevocational class varies 
significantly from other--more standard--classes.3 


A NOTE ON METHODOLOGY 
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person to whom we wish to direct part of our atten- 
tion, comparing him with the student who follows the 
Prevocational curriculum and, in turn, comparing the 
Prevocational pupil to the student in Special Edu- 
cation. Is the central tendency of the Prevocational 
group in Medina, as measured by mean intelligence, 
statistically different from those of Special Education 
and Regent's ? The lack of a statistically significant 
difference between Prevocational and either of the 
other two sections would leave the value of the Pre- 
vocational Program open to question. 


THE MODEL 


In an idealized situation, with intelligence quotient 
as the sole criterion for grouping, the terms of the 
problem are: intelligence quotient cut-off points of 
75 and 90 imply that students with intelligence quo- 
tients below 91 are classed as ‘‘slow learners"; 
those above 90 are deemed able to handle the abstract 
material of Regent's classes; and those below 75are 
*'educable'' in a Special Education class. "Slow 
learners'' enter the Prevocational Program, ''nor- 
mal learners" are placed in Regent's, and **educa- 
bles” enter Special Education. Therefore there are 
по “slow learners" inthe academic section, no “пог- 
mal learners''in the Prevocational Program, and no 
"'educables'' in either of the other sections--the three 
groups are dissimilar on the basis of different mean 
intelligence scores within the groups. The varying 
situations and possibilities may be outlined as follows: 


Special Education and Prevocational 


1. Students with intelligence quotients below 75 
are **educables'' and those with intelligence 
quotients above 74 and below 91 are “slow 
learners. ` 


2. ‘*Educable’’ pupils enter Special Education and 
"slow learners" enter the Prevocational Pro- 
gram. 


3. There are no **educables'" in Prevocationaland 
по **slow learners” in Special Education. 


4. Special Education and Prevocational Program 
students form two distinct groups. 


Prevocational and Regent's 


l. Students with intelligence quotients below 91 


are slow learners" and those above 90 are 
“normal learners, '' 


2. ‘Slow learners" take Prevocational courses 
and ‘normal learners" enter the R egent 5 
program. 

3. 


There are no *'slow learners” in Regent's and 
no **normal learners" in Prevocational. 


4. Regent's and Prevocational Program students 
form two distinct groups. 


The above Syllogisms, re 


states astestab the- 
ses, are in the form: scd 


Hy - Mean of Prevoca 
cial Education 


tional equals mean of Spe- 
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Hy - Mean of Prevocational does not equal mean 
2 of Special Education 


Hy - Mean of Prevocational equals mean of Regent's 


H4 - Mean of Prevocational does not equal mean 
of Regent's 


CONFIDENCE INTERVALS 


The data of Table 2 were used to establishconfi- 
dence intervals for the mean intelligences of Special 
Education, Prevocational, and Regent’s and to test 


the foregoing hypotheses. 


TABLE 2 
SUMMARY DATA 


Mean Standard 
Intelligence Deviation 
Special Education 65. 58 10.26 
Prevocational 87.55 10.12 
Regent's 109.04 23.54 
Advanced 122.04 16. 30 
Prevocational 


85.90 <и < 89, 20 is established as the 95 per- 
cent confidence interval for the mean intelligence of 
the group formed by all Prevocational Students in 
grades 7-12. In other terms, using this research 
methodology, the probability is . 95 that the interval 
85. 90 - 89.20 will contain u , the population mean, 
as estimated by X, a sample meanusedas the point 
estimate of ш. 


Regent's 


107.31 < ц < 110, 76 is the 95 percent confidence 
interval for the mean intelligence of the Medina 
Regent's group in grades 7- 12. 


Special Education 


The establishment of the interval 62, 70 < p < 
68. 50 as containing the intelligence mean of allSpecial 
Education students leads to the next and most impor- 
tant section of the research--a Proof that, on basis 
of mean intelligence, Special Education, Prevoca- 
tional, and Regent's are discrete groups. 


Hy IS PROVED: MEAN INTELLIGENCE OF PRE- 


VOCATIONAL DOES NOT EQUAL MEAN INTELLI- 
GENCE OF SPECIAL EDUCATION 


As a first step in proving that the mean intelligence 
of Prevocational differs from that of Special Educa- 
tion the sample standard deviation ox, - х, will 
be computed by combining the standard deviationsot 
Prevocational and Special Education to arrive ata 
sample estimator. 


= sample mean of Prevocational 


х, = sample mean of Special Education 


1. 68 Standard Deviation 


Now the hypotheses Hy x үе" 
be tested. The critical region for the test is based 
on equal tail areas under a normal Curve with 

œ 2,05. Thus, the critical regionis that part of the 
horizontal axis more than 20 Xi -X = 3.36 units 


x Н, US EN, will 


from 0 (zero) where 0 represents the mean intelli- 
gence of Prevocational -- 87. 55. As the two samples 
of 142 and 50 students yielded the value & 


j^ T 
. 97 falls inthe critical 
2 is rejected, 


87. 55 - 65.58 = 21. 97 and 2 
region the hypothesis x 


The central tendencies of the Prevocational and Spe- 
cial Education groups in grades 7-12, as measured 
by mean intelligence, are Significantly different, 


Hy IS PROVED: MEAN INTELLIGENCE OF PRE- 


VOCATIONAL DOES NOT EQUAL MEAN INTELLI- 
GENCE OF REGENT'S 


Turning to Ha: x = X3 and Hy: Xi я EM the same 
procedure employed in the Prevocational-S p ecial 
Education proof will be used to test these hypotheses: 
by combining the standard deviations of the mean in- 
telligences of the two groups o X, - x, wi 
will be e 
pes g 1 3 Will be estab- 


х 
' 


1 > Sample mean of Prevocational 


ж 
" 


з 7 Sample mean of Regent's 


= 1. 26 Standard Deviation 


The critical region is based on equal tailar 
the normal curve for xi - E with о = 95 


Part of the horizontal axis more than 2 о А 
"Д 
2. 52 units from Zero where Q 
ч 15, ар; 
telligence of the Prevocationa] groan 8 
the ey Samples of 142 and 630 Students yieldit 
1 Хз = 87.55 - 109, 04 = 21, 49 the hypothe- 
sis X =X, 


3 15 rejected as 721.49 falls in the criti- 


cal region, 
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E -2.52 2.52 


The central tendencies of the Prevocational and Re- 


gent's groups in grades 7-12, as measured by mean 
intelligence, are significantly different. 


EVALUATING THE PROGRAM 


Research on the Medina Prevocational Program 
has been largely aimed at evaluating the effects of 
the program on the Students. Two of the more im- 
portant studies are Nunzio T. Maiorana ‘‘Compara- 
tive Analysis of Attitudes, ” and George C. Barna 
“The Holding Power of the Prevocational Program.” 


Nunzio Maiorana, ina long-term study, hascon- 
centrated his attention on determining whether the 
attitudes of Prevocational students change after en- 
tering the program (3). In the study he deduces at- 
titudes by observing the choices students make in 
their overt behavior. Maioranauses the sixth grade 
records of Prevocational students as his base, and 
finds that after entering the program attendance is 
up 54 percent; the amount of homework turned in ap- 
proximates the percentage turned in by a random 
sample of students in academic classes; distribution 
of grades compares closely with academic classes; 
and the number of behavior problems, as determined 
by examining anecdotal records, are down 60 per- 
cent. 


George Barna, prior to the graduation of the first 
Prevocational class, compared the dropout rate of 


students with intelligence quotients of less than 95 
who were in regular classes to the dropout rate of 
pupils in the Prevocational Program (1). He ascer- 
tained that the Prevocational Program exerted a 
holding power 12 percent greater than other programs 
in the system. That is, a 20 percent dropout rate 
in Prevocational as compared to a 32 percent drop- 
out rate for a comparable group of students who were 
in other school programs, Mr. Barna is now in 
the process of comparing the dropout rate of the Pre- 
vocational Program to the statewide rate for students 
with borderline intelligence quotients, For the pur- 
poses of his study an intelligence range of 75-95 is 
being used. The updated studies of Maiorana and 
Barna will appear, along with other descriptive 
Statistics, in the “Prevocational Program in Grades 
7-12 of Medina Central School, Sixth Progress Re- 


port, ” copies of which can be obtained from the 
Medina Central School System. (6) 


THE APPLICABILITY OF THE STUDY 


What is to be gained from an isolated study of this 
nature? Results can be generalized to encompass 
a larger population and geographic area. It can be 
assumed, on the basis of other Studies, that **slow 
learners” are normally distributed and comprise 
17-20 percent of the general population so, approxi- 
mately these same results will hold for samples of 
normally distributed populations if selection criteria 
are held constant, Thus, educational facilities which 
draw their enrollment, in an unbiased manner, from 
a representative section of the general population, 
will contain a proportionate number of ‘‘slow learn- 


ШЕ. 


ers” within the schools involved. If this hypothesis 
holds true then it becomesthe responsibility of school 
systems to consider designing curricula tailored to 
the needs of ‘‘slow learners" who, on the basis of 
mean intelligence, form a group midway in a pro- 
gression between Special Education and academic 
classes. 


FOOTNOTES 


l. Results on tests of mental and reading ability are 
easily quantifiable while assignments based 
on ‘‘limited background''are extremely sub- 
jective. Ina public school system, where 
it is impossible to examine a student's home 
life in detail, an estimate of deprivation must 
be made. In the majority of cases, however, 
there is a colinear relationship between two 
or all of the factors. 


2. From results gathered during an experimental 
program of group remedial reading for the 
**slow learner” it was found impossible 
to predict the amount of reading disability 
or a possible increase in reading ability on 
the basis of measured intelligence. 


3. Selection procedure determines the nature of any 
group. In the case of the Prevocational Pro- 
gram the assigned cut-off points of 75 and 
90 determine that, almost assuredly, the 
mean intelligence of the group will fall in 
this range. Whether or not the mean intel- 
ligence is significantly different from that 
of other groups must be determined statis- 
tically. 


4. An “Advanced” student is a pupil having one or 
more classes in which the content extends 
beyond the state- mandated minimum require- 
ments. 
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GENERATING A PROJECTED CRITERION OF 
GRADUATE SCHOOL SUCCESS 


VIA NORMATIVE JUDGMENT ANALYSIS' 
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Institute for Development of Educational Activities 


ABSTRACT 


This study was designed to investigate the extent to Which a policy regarding graduate admission Standards 
exists among selected graduate faculty members at Colorado State College. Twenty judges utilized Normative 
Judgment Analysis techniques to generate a projected criterion of graduate school success on profile data for 
thirty randomly selected doctoral graduates. The results of the study indicate that essentially one policy was 


being expressed by the judges. 


4 SEARCH of the literature reveals that the num- 
ber of published studies in predicting graduate 
School success is small and nearly all sample sizes 
In general, the studies 


A potentially useful technique which avoids the 
criticisms resulting from using the GPA as the de- 
pendent variable is Normative Judgment Analysis, 


ity of their prediction equations. This approach is, 
in turn, a special application 9f a still more general 
hierarchial grouping developed by Ward (3). Norma- 
tive Judgment Analysis (JAN) is a multiple linear 
regression approach used to determine the relative 
importance of the various profile items for each 
judge in the expression of his policy. In the case of 
admitting students to graduate school, it allows the 
admitting school to make a decision whether to ac- 
Сері a given student on the basis of a policy which 
has been expressed by JAN. The policy is a state- 
ment of the Weights to be applied to Scores obtained 
on the Various profile items for each student seek- 
ing admission, It clearly identifies a group position 
on the relative importance of the various profile 
variables utilized in making decisions about the ad- 
mission of potential Students. 


ing the usual procedure of asking each judge to de- 
termine the relative importance of each of the pro- 
file items so that a proper weighting of the variables 


and give him an overall Score that will serve 
rank the applicants in an order of merit, 


Stimuli and consequent]: Obtai i 
for each judge. A iiis 


Once these equation. 
tained from all the ju 


equations, the crite- 


is employed to define 
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areas of agreement and disagreement among 
the board members. As the first step, a sin- 
gle value of В? is computed which indicates 
the overall predictive efficiency obtained when 
a separate least-squares-weighted regression 
equation is used for each rater. During the 
second step, each equation is compared with 
every other equation. In this way, the two 
raters are located who are in closest agree- 
ment concerning how the selection variables 
should be weighted to determine the relative 
acceptability of applicants. These two raters 
have the most homogeneous regression equa- 
tions. The computer then determines the sin- 
gle equation that best represents the joint 
policy of these two raters. It also indicates 
the overall loss in predictive efficiency that 
results when the original N equations are re- 
placed by the best set of N - 1 equations. This 
is accomplished by comparing the magnitude 
of the overall R* in the two instances. 


In subsequent operations, the program sys- 
tematically reduces the number of prediction 
equations (or rater clusters) by one at each 
Step so as to minimize the loss in overall pre- 
dictive efficiency. In each instance, the com- 
puter prints out an overall R?, as well as the 
single equation for each cluster which best 
represents the joint policy of all raters in that 
cluster. At the last step, a single prediction 
equation is generated so as to express the 
joint policy of all raters with as little error 
as possible. 


Examination of the loss in overall predic- 
tive efficiency at various stages of the group- 
ing process will help to identify the number of 
rating policies existing among board members. 
For example, if very little overall predictive 
efficiency is lost in going from ten to two equa- 
tions, but a considerable amount of efficiency 
is lost in going from two to one equation, there 
is Strong evidence that two different rating pol- 
icies are operating. That is to say, some of 
the board members are using one weighting 
system and the rest a different weighting sys- 
Y tem, Of course the hope is that a single equa- 

tion can be found which predicts the criterion 
values assigned by all raters. In this case the 
board could be dismissed without further ado 
Since their single agreed-upon policy would | 
have been determined, However, should more 
than one rating policy be found to exist, the 
board is not asked to arbitrate the difference 
їп weights. Rather, the several equations are 
used to rank the applicant sample, and the 
board is asked to arbitrate the differences in 
the rank position of cases resulting from appli- 
cation of these Several equations. Once a sin- 
gle agreed-upon rank-ordering of the sample 
has been obtained, it is a simple matter to de- 


termine the appropriate set of weights for the 
final selection composite. 


The JAN technique enables the invest or 
q S the investigati 


plete analysis of in terrater 
ене As a matter of fact, results of the 
ret stepalone indicate the amount of agreement 
between each rater and every other rater (2: 4) 


о “f 
T 


The JAN procedure seems to have numerous ad- 
vantages recommending its use. The most impor- 
tant is that it enables the investigator to capture the 
policy of a judge in a very clear and methodical 
manner. In addition, it is objective" in the sense 
that each judge is asked to render a decision based 
on available data rather than on individuals or per- 
sonalities. Further, JAN permits the use of non- 
linear terms in the regression model which means 
it is essentially unrestricted mathematically. Fi- 
nally, another obvious advantage is that its ba sic 
judgment is probably a much more realistic assign- 
ment for judges than the typical situation which re- 
quires the judge to ascribe the various weights to 
each predictor variable. 


DESCRIPTIONS OF THE VARIABLES (Table 1) 


ion 


1. Normative-JAN crit 


The rating of each of the Ss was pentamerous 
coded, 1 identifying those students whose projected 
graduate success placed them into the bottom 20th 
percentile of successful graduate students at Colo- 
rado State College; 2 identifying those students 
whose projected graduate success placed them into 
à percentile between the 20th and the 40th; 3 identi- 
fying those students whose projected graduate suc- 
cess placed them into a percentile between the 40th 
and 60th; 4 identifying those students whose projected 
graduate success placed them into a percentile be- 
tween the 60th and the 80th; 5 identifying those stu- 
dents whose projected graduate success placed them 
at or above the 80th percentile. 


2. Colorado State College master's degree 


Whether or not a master's degree was received 
from Colorado State College was binary coded for 
each of the Ss, 1 identifying those students who ob- 
tained their master's degree from Colorado Sta te 
College; O identifying those students who received 
their master's degree from some School other tban 
Colorado State College. 


3. Cumulative grade point average 


Cumulative grade point average was recorded on 
a five- point scale, 5 points identifying an A; 4 points 
identifying a В; 3 points identifying a C grade;2 
points identifying a D grade; 1 point identifying an F 
grade. It included all bachelor's and master's de- 


sree work taken previous to undertaking doctoral 
degree studies. 


4. Years between B. A. and M. A. degree 


5. Rating on Interview 


The rating on the inte 
on a four-point scale; 
most unlikely to succe 


rview of the Ss was recorded 
1 identifying those students 
ed; 2 identifying those Students 
who are doubtful; 3 identifying those Si uden T чы 
are likely to succeed; 4 identifying those st А 
who are most likely to succeed. This ratin 
Sents the concensus of the graduate 
pointed to interview each applicant f, 
degrec program. 


repre- 
committee 4 p- 
Or the doctora] 


6. Recommendation on Interview 


Codea 1 if the S 
the doctoral degree 


was recommended to 
program, i 


continue 
other wise о ела 


TABLE 1 
LIST OF VARIABLES 


Number Variable Abbreviation 
1 Normative-JAN criterion NJ 
2 Colorado State College CSCM 
master’s degree 
3 Cumulative grade point CuGPA 
average 
4 Years between B. А. and YrBD 
M. А. degree 
5 Rating on interview Ral 
6 Recommendation on interview Rel 
Т: Sex Sex 
8 GRE-Verbal GRE-V 
9 GRE-Quantitative GRE-Q 
10 GRE-Social Science GRE-SS 
11 GRE-Humanities GRE-H 
12 GRE-Natural Science GRE-NS 
13 GRE-Advanced Education GRE-Ed 
14 English Usage Test score EnU 
15 Education major EdM 
16 Science major SM 
ч Psychology major PM 
18 Business major BM 
19 Music major MM 
20 Industrial Arts major InM 
21 Physical education major PEM 
22 Other major field OtM 
7. Sex 


Coded 1 is the S was a male, O for a female, 


8. Graduate Record Examination — Verbal score 


9. Graduate Record Examination 


- Quantative 
Score 


10. Graduate Record Examination 


= Social Science 
Score 


11. Graduate Record Examination - 


Humanities 
Score 


12. Graduate Record Examination - N 


atural Sci- 
ence score 


13. Graduate Record Examination — Advanced Edu- 
cation score 


14. English Usage Test score 


The English Usage Test is a special test devel- 

Ped and standardized at Colorado State College de- 

Signed to measure written communication sk i 11 
Results were reported as percentile scores, 


s. 


15. Education Major 
-n Major 


Coded lif thi 


е S was an education major, other- 
wise О. = : is 
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16. Science Major 


Coded 1 if the S was a science major, other- 
wise O. 


17. Psychology Major 


Coded 1 if the Swasa psychology major, other- 
wise O. 


18. Business Major 
Coded 1 if the S was a business major, other- 
wise O. 


19. Music Major 

Coded 1 if the S was a music major, otherwise O, 
20. Industrial Arts Major 

Austria! Arts Major 


Coded 1 if the S was an industrial arts major, 


otherwise O. 


21. Physical Education Major 


Coded 1 if the S was a physical education major, 


otherwise O. 
22. Other Major Field 
Ier Major Field. 


Coded 1 if the S was a social Science, English or 
general plan major, otherwise O. 


PROCEDURES 


Twenty judges, representative graduate faculty 
members from Colorado State Colle ge, were 
requested to respond to the protile data on 30 ran 
domly selected doctoral graduates fro m Colorado 
State College (1 963-1966). Ea ch judge was asked 
to place each student on the basis of the Profile in- 
formation into one of five Categories. A sc Ore of 
five indicated that the judge thought the Student 
would probably graduate in the top fifth of Colorado 
State College graduates, Similarly, a Score of one 
indicated a judgment that the student would graduate 
in the lowest fifth of Colorado State College gradu. 
ates. The purpose was to ascertain indi rec ily a 
group policy on the part of the faculty inte rms of 
the predictor Variables, Sixteen or 80 perce i 
the judges responded, ini 


RESULTS AND DISCUSSION 


A multiple linear regression e 
oped utilizing the data obtained fror 
р а а m the 16 res 
with 21 predictor variables to Predict a Meee 
variable of ranked graduation Category 
ent analysis Was restricted to к 
questions: a) the 
tem for predicting the ranked yy 
and b) the efficienc dicti peer 
y of the prediction wh 
EA = еп certain 
g PS of predic from the system 
5 hree groupg of predi 
bles Investigated were i a) варь o 
> = S 
); b) Test data (variable 


©) Major subj ia 
15-22 inclusive). Jor subject Variables (va. 


quation was devel- 


varia- 
Ms (үар- 
88-14 jy- 
riableg 
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TABLE 2 

INTERCORRELATIONS 

Variable 1 2 3 4 5 6 т 8 9 10 n 12 13 14 15 16 17 18 19 0 21 22 
NJ 1 

CSCM2 -37 

CuCPA 3 -10 31 

YrBD4 -01 -21 -08 

Ral 5 05 от 09 30 

Rel 6 01 19 03 -16 06 

Sex 7 04  -33  -10 12 -04 -22 

GRE-V8 38 17 36 -08 27 09 -19 

GRE-Q9 27 -32 31 -53 -25 35 -12 36 

GRE-SS 10 53 11 27 -16 09 12 -09 76 46 

GRE-H11 16 03 27 -30 04 21 -45 56 22 42 

GRE-NS 1249 20 29 -2) 13 24 -06 75 61 82 49 

GRE-Ed 1316 28 29 -10 31 31 -25 67 25 64 30 54 

EnU14 37 00 o 12 18 20 -24 58 17 47 46 49 26 

EdM15 09  -05 27 13 22 21 -10 14 12 33 n 14 40 02 

SM 16 11 17 12 15 28 13 05 20 05 14 12 34 04 10 13 

PM17 09 2] -09 -63 -17 30 -11 25 31 18 27 27 30 01 30 -12 

BM18 -59 31 19 1 18  -17 -14 -24 -28 -39 -11 -33 -21 -05 -17 205 16 

MM19 -66 18 35 08 -26 00 -06 -22 -22 -39 18 -34 -28 -19 00 -05 -17 37 
InM 20 -3I -03 -05 91 204 -01 01 -07 -03 -27 -10 -14 -22 00 -01 -05 -18 14 18 
РЕМ 21 -1 45 09 10 -05 00 -06 -23 -12 -36 -25 -34 -01 -1) 00 Bu 38 47 18 
OtM22 -83 -95 зв -06 01 22 -17 -07 -03 


-03 58 72 33 72 


The intercorrelation coefficients (Table 2) among 
the 22 variables are product- mo ment coefficients. 
Because some of the variables are binary coded, 
some of the coefficients are phi'S and some are 
point-biserial's. 


The predictor variables were selected because 
they represented readily accessible ty pes of infor- 
mation which, it was believed, might have some pre- 
dictive relationship to the policy expressed by the 
judges. The coefficients in column 1 of Table 2 show 
these relationships. Reference to Table 2 shows 
that the three highest zero-order correlations be- 
tween predictor variables and the criterion are Oth- 
er Major Field (-. 83), Physical Education Major 
C. 71), and Music Major (-. 66). 


The investigator use 
according to Ward (4; 204-237), to determine the 
unique contribution 
variables, (2-22), 
Normative-JAN rating. 


The hypothesis tested stat 
that these variables co; Es 


mination of the expect 
already available in tl 


If a variable i i 
bution, then know! e is makin, 


formation about the criterion. Also, if a variable 
is making a unique contribution, then two Ss, who 
are unlike on the variable but who are exactly alike 


or are matched on the other predictors, will differ 
on the criterion. 


The results of the regression analysis are sum- 
marized in Tables 3 and 4. Table 4 indicates that 


TABLE 3 


MEANS AND STANDARD DEVIATIONS (N = 30) 


Standard 

Variable Mean Deviation 
1 NJ 2, 80 2.15 
2 CSCM .48 .58 
3 CuGPA 3. 96 .46 
4 YrBD 7. 99 3.59 
5 Ral 1.87 .76 
6 Rel ‚61 48 
1 Sex . 93 . 24 
8 GRE-V 491.31 92. 78 
9 GRE-Q 498. 86 115. 96 
10 GRE-SS 523. 75 85. 00 
11 СВЕ-н 511.96 16.13 
12 GRE-NS 514. 20 100. 52 
13 GRE-Ed 551. 06 78.81 
14 EnU 65.32 25. 83 
15 EdM .33 47 
16 SM .03 к 
17 PM +30 als 
18 BM . 08 n 
19 MM *11 31 
20 InM 107 28 
21 РЕМ 111 26 
22 ом 01 +43 

.27 
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TABLE 4 


PROPORTIONS OF VARIANCE ATTRIBUTABLE TO GROUPS OF VARIABLES BELIEVED TO BE 
ASSOCIATED WITH NORMATIVE JAN CRITERION 


PREDICTOR-Variable Total Contribution Unique Contribution Multiple Correlation 
Group Proportion (RS) Proportion with Criterion 
b 
Model 1 (2-22) . 8580 -935 
Model 2 (2-14) . 5928 16 
Model 3 (2-7, 15-22) .8127 ‚90Р 
Model 4 (8-22) . 8203 .91b 
Major Subject Field (15-22) .2652* 
Test Data (8-14) . 04533 
Biographical Data (2-7) .0377* 


ад proportions reported as unique contributions are Significant at the . 01 level for N - 480. 
In computing F values, it was assumed that one parameter was associated with each variable 
in the prediction system. The degrees of freedom for the number of predictors were determined 
by the number of variables given an opportunity to contribute to the prediction, 


Psigntticant at the . 01 level, 


the 21-variable composite accountsíor . 5580 of the field data. The difference between RS value for the 
total criterion variance which is significant at the FM system (. 8550) and the RS value for the restricted 
‚01 level A multiple correlation using the 21 var- model, FM - (8-14), yieldsanestimate of . 0453 for 
iables as predictors of the Normative-J AN rating the unique contribution of variables (8-14) asa 
уаз. 93. This is not only significant at the . 01 ley- group. The loss in predictive accuracy when the 
el, but it is exceptionally high from a predictive test data (8-14) are not considered seems sufficient 
point of view as approximately 86 percent of the to be of practical importance, The unique contribu- 
variance is accounted for when the 21 variables are tion of the seven predictor variables involving test 
used as predictors. The high RS (. 8580) indicates data information (8-14) was significant at the ‚01 
that the interrater agreement is exceptionally high level. 

and that there is essentially one policy being ex- 

pressed by the judges using the Nor mative JAN In model 4 a 15- variable composite (8-22) was 


technique. tested for predictability in which six variables (2 7) 


In model 2 а 13- variable composite (2-14) was omitted as predictors. About 82 percent (. 8203) of 
tested for predictability in whichthe eight variables the criterion variance is estimated tobe attributabl 
(15-22) which identified the major subject field of to the 15 variables which essentially consist oft t: 
doctoral study were omitted as predictors. About and major subject field data. The differ ps 


59 percent (. 5928) of the criterion variance is esti- tween RS value for the FM System (. 85 
mated to be attributable to the 13 variables whic h 
essentially consist of biographical items andtest 
data. The oo earn between RS value for the FM 
system (. 8580) and the RS value for the restricted { = 

node, FM - (15-22), yields an estimate of . 2652 составу When the biographical data (2- 
for the unique contribution of variables (15-22) asa 

group. The loss in predictive accuracy when major 
Subject field data are not considered seems suffi- 
cient to be of practical importance, The unique 

Contribution of the eight predictor variables (15-22) 
involved with identifying the major subject field was 
Significant at the . 01 level. 


In model 3 a 14-variable composite (2-7, 15-22) 
Was tested for predictability in which seven vari- 


ables (8-14) which yielded test data information i gator w 
: е in determining to w as mo intereste 
rop ittea as predictors. About 81 percent pressed than t i E extent a policy haq xen e 
. 81; ) of the Criterion variance is estimated to be the RS's only we ual regression equation 
attributable to the 14 variables whi ch essentially to give an exhau = obtained, attempt was S, and 
consist of biographical items and major subject variables a ave analysis of the Set of pr quide 
d ea edictor 


5 o 
» the Predictor variables Were 
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grouped into three proper disjoint subsets and the 
unique contribution of each of these subsets was ex- 
amined. 


FOOTNOTES 


1. The material in this article is part of a larger 
study submitted in partial fulfullment of the 
requirement for the degree of Doctor of Phi- 
losophy at Colorado State College, 1967, 


2. Currently the author is a Research Specialist, 
Research Program, Institute for De velop- 
ment of Educational Activities, a Charles F, 
Kettering Foundation project. 
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THE RELATIONSHIP OF COPING STRENGTH 
TO SELF-CONCEPT, SCHOOL ACHIEVEMENT, AND 
GENERAL ANXIETY LEVEL 
IN SIXTH GRADE PUPILS 


THOMAS M. HUGHES 
Memphis State University 


ABSTRACT 


The primary purpose of this study was to determine the predictive quality of re: 
comprehension coping strength measured under conditions of delayed auditory feedback 
concept, anxiety, reading achievement, intelligence, and academic abilit 


feedback were administered to fifty-one 
Tennessee. Subjects were chosen from five elen 


was defined as reading coping streng 


of self-concept, anxiety, reading achievement 
grades. 


THE DISPARAGING effects of delayed auditory 
feedback have been substantially investigated. Ina 
reprint of his article which appeared in a speech 
journal, Chase (2) listed over one hundred articles 
and related papers on the subject. Concurrent with 
the work of Chase, considerable work has been go- 
ing on in delayed auditory feedback at the University 
of Illinois under the direction of Mowrer. Very re- 
cent research on delayed and accelerated auditory 
feedback has been produced at the Neurocommuni- 
cations Laboratory of the John Hopkins Hospital in 
Baltimore, Maryland. This work is also under the 
direction of Chase. 


When a subject speaks into a system that delays 
the auditory feedback of what the subject has said, 
it creates two distinct phenomena, i.e. , increased 
vocal intensity and stuttering- like disfluences, For 
Some, these effects are associated withan inability 
to concentrate on the speech content. The effects of 
delayed hearing of one’s voice as he speaks were re- 
ported by Lee (5), and have since been repeated in 
many studies. Conversely, other subjects tend to 
build a gradual resistance to the disrupting effects 


to the point that they develop a near mastery of the 
phenomena, 


Kerstin and Eysenck (4), in studying pursuit ro- 


sa 


ading coping strength and 
as they relate to self- 
y. Readingtests under delayedauditory 


sixth-grade students in the Memphis City School System, Memphis, 


mentary schools ( three white and two Negro ). Reading error 
Scores, under conditions of DAF, were tabulated as a measure of readin; 


coping strength. Eight hypotheses were established and tested by ANOV. 
relation. Overall results of the study tend to identify coping strength 
psychology of reading as well as the psychology of such noncognitive va 

The problem of this study was to make a systematic investigation of the abilit 
level to cope with the effects of delayed auditory feedback while reading. 


g coping strength and comprehension 
A, t-test, correlation and multiple cor- 
às a unique variable to consider in the 
riables as anxiety and self-concept. 

y of childrenatthe sixth grade 
The ability to cope with such effects 


th and comprehension coping strength. These two variables, reading coping 
Strength and comprehension coping strength, were studied todetermine their 

and to compare their predictive strength with that of IQ and re; 
the problem of this Study was to determine the intervariable and 


ability to predict academic success 
ading achievement scores. An additional part of 
composite relationships between the variables 


‚ reading coping strength, comprehension coping strength, IQ and 


tor performance as a function of different degreesof 
auditory distraction, came to the following conclu- 
sions: (1) distraction always lowers performance, 
even when the signal rate is very slow; (2) perform- 
ance decrement is proportionalto the amount of dis- 
traction, there being a linear relation between per- 
formance decrement and signal rate; (3) the amount 
of performance decrement is in part determined by 
individual differences, some Ss's being more dis- 
tractible than others. 


In the research by Baker and Madell (1) there is 
found the strongest rationale for continued Study of 


eee coe of distraction on certain personality var- 
iables: ; 


The results of the 


present study also illus- 
trate clearly the prin A [us 


rly ciple that personality 
characteristics which are not operative and 


determinative of behavior under relatively 
benign Circumstances may become So under 
more stressful conditions, This finding h 
Important implications for i oe 
Sitivity of psychodiagnostic 
eral, but in Particular it su; 


ү а reading cor 
prehension task to redict z i Ya 

c 2 c Я 
EE. р: t academic perform- 


60 THE JOURNAL OF EXPERIMENTAL EDUCATION 


It is believed by this writer thata correlation may 
exist between the ability to cope with the effects of 
delayed auditory feedback while reading and academic 
ability. Itisalsobelievedthat the variables of self- 
concept and anxiety levelare related to reading coping 
strength, comprehension coping strength, andacadem- 
ic ability. These assumptionsare supported by Spilka 
(7) and Rosenfeld (6). 


GENERAL DESIGN 


This study investigated the performance of sixth- 
grade Negro and white children interms of their re- 
sponses to delayed auditory feedback distraction as 
a measure of coping strength. Coping strength was 
dichotomizedas reading coping strength and compre- 
hension coping strength, Reading coping strength 
was determined by the child’s ability to maintain vigi- 
lance during distraction by delayed auditory feedback, 
thus keeping hiserrors of omission, substitution, and 
pronunciation at a minimum. Comprehension coping 
strength was determined by the child’s ability to main- 
tain vigilance in order to retain material read under 
the condition of delayed auditory feedback, and to keep 
errors ata minimum ona testof comprehension. 


concept, anxiety, academic achievement, andread ing 
comprehension, 
attempted to contribute to the development of a tool 
by whicha classroom teacher, educational psycholo- 
gist, or counselor may study the potential for academ- 
ic ability on the basis of a child's performance ona 


METHOD 


Fifty-one sixth-grade children 
Negro--26 girls and 25 boys) from five elementary 


data sheets, IQ range was confined to 89-112 (sta- 


The investigator tested the 
mor 


nings in the School cafeteria of tl i 
к, E of the school which 


recorder to become acquainted with the effects of 
DAF while reading. After the practice period, individ- 
ual administration of the Gray, Durrell, and Spache 
reading tests was accomplished under DAF conditions. 
Each test was scored for time, errors of omission, 
substitution, and mispronunciation. As soonasa child 
finished a reading test on the tape recorder, he went 
to an isolated place to take the comprehension test un- 
der a proctor, andwas Subsequently scored for errors 
of comprehension. While the individual tests were 
being given, the other children were removed to an- 


other area so they couldnot hear the reading or inter- 
fere with the child being tested. 


Each child’s level of coping strength was deter- 
mined by the total number of errors made on the three 
reading tests under the DAF condition and the total 
number of errors made on the tests of comprehen- 
sion. Thus, the lower number of errors made on the 
reading tests indicated higher reading coping strength. 
In the same manner, the lower number of errors 
made on the comprehension tests determined higher 
comprehension coping strength and the higher num- 


ber of errors made determined lower comprehension 
coping strength. 


STATISTICAL PROCEDURE 


Midterm grades were obtained and converted in- 
to numerical equivalents ( A4, B3, C2, D1, F0). 
ed with the variables of 
self-concept, anxiety, and 
able correlations were de- 


RESULTS 


Hypothesis 1. 


Тһеге їз по Significant 
two coping Strength score. 


tion of IQ and reading ai 
diction of a 


Dus variables of read 
Prehension copin; st 
ere ping strength were placed in multiple 
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TABLE 1 


A COMPARISON OF THE MULTIPLE CORRELA- 
TION OF READING COPING STRENGTH AND COM- 
PREHENSION COPING STRENGTH WITH GRADES 
AND IQ AND READING ACHIEVEMENT WITH 
GRADES 


r Standard z Test of P 
Error Significance 


RCSandCCS* -.660 4. 06 


IQand Reading 
Achievement . 537 4.11 1:38 2.05 


*Reading coping strength and comprehension coping 
strength as number of errors under DAF condition. 


of coping strength. Those children with higher intel- 
ligence made fewer errors in reading and compre- 
hension than children with lower intelligence. There- 
fore, children with lower IQ scores who were making 
poorer grades had much higher error scores than 
brighter children who were making good grades. 


The coping strength variables yieldeda somewhat 
greater predictor г strength than IQ and reading 
achievement. The 2 test of significance between the 
two correlations is shown in Table 1. 


Hypothesis 1 is accepted since p = <. 05 level of 
confidence was not reached. However,the.05 level 
of confidence was approached, and thus it appears 
that reading coping strength andcomprehension cop- 
ing strength are highly comparable with the variables 


TABLE 2 


INTERCORRELATIONS OF READ: 


of IQ and reading achievement in the prediction of 
grades, 


Hypothesis 2 


There is no significant difference between read- 
ing coping strength and comprehension coping strength 
in the prediction of grades, 


Data in Table 2 indicated that the comprehension 
coping strength correlation of -. 622 was higher than 
the reading coping Strength correlation of -. 498. 
This appears to lead to the inference that a child’s 
ability to retain ideas or concepts while coping with 
the distracting effects of delayed auditory feedback 
is a better predictor of his academic performance than 
his ability simply to reduce his reading error score 
while coping with distraction. Apparently the ability 
to cope with distraction while formulating concepts 
in reading is more facilitating than simply inhibiting 
the mispronunciation and omission of words. 


Of additional interest in the matrix of Table 2 is 
the interrelationship between the two measures of 
coping strength and the variables of grades, self- 
concept, and anxiety. The two measures of coping 
Strength showed a significant correlation of . 491 with 
each other, However, comprehension coping strength 
showed a much stronger relationship with self-con- 
cept. This may be construed to mean that children 
with stronger self- images were able todeal more pro- 
ficiently with concepts while under distraction than 
to control reading error rate. Both measures of 
coping strength showed low and not significantly dif- 
ferent correlations with anxiety. 


Hypothesis 2is accepted on the basis that the dif- 
ference between comprehension coping strength and 
reading coping Strength prediction of grades, as shown 
in Table 3, did not reach the - 05 level of significance, 
Significance was approached with p = <, 13, 


ING COPING STRENGTH, COMPREHENSION COPING STRENGTH, GRADES 
SELF-CONCEPT, AND ANXIETY , 


1 2 4 5 6 7 

1. Reading coping 

strength* .491 -.498 =. 182 183 -.569 -.464 
2. Comprehension 

coping strength* -.622 -.479 AT ~. 483 -.334 
3. Grades 411 ~. 108 527 412 
4. Self-concept -. 462 . 099 113 
5. Anxiety . 064 165 
6. 19 

«641 

7. Reading 

achievement 
*Coping strength was the number of error. 
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Hypothesis 3. 


There is no significant difference between selí- 
concept scores of those children who demonstrate high 
coping strength and those who demonstrate low cop- 
ing strength. 


It was originally anticipated thatc hildren with 
high coping strength would show high self-concept 
scores, and children with low coping strength would 
show low self-concept scores. No significant differ- 
ence existed in the self-concept scores of the high 
and low coping strength groups. Reading coping 
strength was the lesser of the two coping strengths 
in the approach to significance. The self-concept 
scores in the high comprehension coping strength 
group approached, but did not attain, the . 05 level 
of significance. Hypothesis 3 is accepted because 
Statistical significance was not obtained. 


TABLE 3 


TEST OF SIGNIFICANCE BETWEEN NON-INDE- 
PENDENT CORRELATION COEFFICIENTS 


—————Є———————Є 


CCS* RCS** RCSwithCCS Significance p 


r -.622 -.498 .491 t--L13 <,13 


*Comprehension coping strength with grades. 
**Reading coping strength with grades. 


Hypothesis 4 


There is no significant difference in grades be- 
tween high and low self-concept groups and high and 


low IQ groups nor any interaction between self-concept 
and IQ. 


Analysis of variance data are presented in Table 
4 which supports rejection of Hypothesis 4. The high 
IQ group earned significantly higher grades than the 
low IQ group. In addition children with a high self- 
concept tended to earn higher grades than children 


with low Selí-concepts, Interaction effects were not 
observed, 


Hypothesis 5 


There is no significant difference 


I S between the 
anxiety levels of low 


and high IQ groups. 

_ Hypothesis 5 is accepted. Data from t-tests in- 
dicated children with high IQ's showed a Slightly 
higher but not st: 


atistically different level of anxiety 
when compared to the low IQ group. 


Hypothesis 6 


There is no signif 
reading COping stren 
above the mean 
the mean. 


ts 


Hypothesis 6 is rejected. Children withIQ meas- 
ures above the mean of 101.56 showeda significantly 
lower number of errors when compared with low IQ 
group. This may be interpreted to mean that child- 
ren with higher IQ's may be expected to cope better 
with distraction and to make significantly fewer er- 
rors in readingand comprehension under the condi- 
tion of delayed auditory feedback. 


TABLE 4 


ANALYSIS OF VARIANCE OF GRADES BY IQ AND 
SELF-CONCEPT (MEAN SPLIT OF GROUPS) 


Source SS df MS F P 
IQ 186.06 1 186.06 7.47 .01 
Self-concept 96.24 1 96.24 4.86 · .10 
IQ x SC 18.74 1 18.74 “75 
Еггог 1171.08 47 24. 92 
Total 1472.12 50 
Hypothesis 7. 


There is no significant difference between Negro 
and white students on measures of coping strength, 
anxiety and self-concept. 


Hypothesis 7 is rejected because reading coping 
strength of Negro pupils was statistically superior. 


Significant differences did not exist between Negro 
and white students on measures of the variables of 
self-concept and anxiety. The white students showed 
less anxiety and more positive self-concepts, but the 
level did not approach significance, 


Hypothesis 8 


There is no significant diff 
and girls on measures of со 
and self-concept. 


erences between boys 
ping strength, anxiety, 


Hypothesis 8 is accepted. Reading coping strength 
did not vary significantly between boys and girls. 
Self-concepts did not vary significantly between boys 
and girls although girls had a slightly higher mean 
self-concept. Boys had greater comprehension cop- 
ing strength than girls. The anxiety level of girls 


was higher than boys with the difference approachin 
significance. 8 


PREDICTING GRADES FROM COPING STRENGTH 


у Garrett (3) 
™MPo site total of 
ementary school: 


eight grades which are given in el 
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Y pred. -r2Y-. Х__ гоу. Mx+ My. 
Ux ox 


г = correlation between X and Y. 

X - reading coping strength score. 

Y = numerical equivalent of eight 
subject grades. 


oy and cx =the standard deviations of the X 
and Y scores, 
Mx and My = the means of the X and Y scores. 
Y pred. = the predicted score of grades. 


Computing this formula we find: 
Y pred. =-. 50x 5.477 X__-. 50x 5.477x 22.78418.35 
14. 036 14. 036 


and 
Y pred. = . 19X + 13. 91. 


If, for example, a student made seven ( number 
of errors ) on the reading coping strength test, his 
most likely composite of grade total would equal 32 
or A average. This is accomplished by subtracting 
the number of errors he made from 100, then apply- 
ing the predictor formula. 


100- 7= .93 
.19 
17,67 + 13.91 = 31. 58 


From a composite of eight grades on the five point 
system (A-4, B-3, C-2, D-1, F-0) it may be seen 
that a student with a reading coping strength of seven 
should have a grade average of A. The lower the 
reading coping strength error score, the higher we 
should predict his grades. The same would hold true 
for comprehension coping strength. 


In the application of such a formula, it should be 
understood that grade meaning varies among schools, 
An A in one school does not necessarily mean that 
the same standards were met in another school bya 
student who made an A, Therefore, should a coping 
strength score be usedfor prediction, the standards 
of the individual school would have to be taken into 
consideration. 


PREDICTING GRADES FROM READING TESTS 


Before each child was administered the coping 
strength tests, he was given three timed paragraphs 
to read. These paragraphs were from the Spache, 
Durrell, and Gray tests and were administered free 
of the conditions of delayed auditory feedback in or- 
der to observe how well they predicted grades in com- 
parison with material read under the conditions of 
delayed auditory feedback. Table 5 shows that the 
error scores on the combined tests predicted grades 
at -. 94r and the comprehension portion predicted at 
-3ir. When these results are compared to the cor- 
relations in Table 2, itis seenthat the reading tests 
accomplished without delayed auditory feedback had a 
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slightly higher r thanthose under the DAF conditions 
(-.498). However, the comprehension tests under 
DAF (-.622) showed a much stronger correlation 
than the -31r of tests without DAF. These results 
appear to point out that children who can maintain 
greater viligance or attention under the condition of 
DAF retain more of the material read, although read- 
ing error rate does not vary significantly from one 
level of vigilance to another. 


In general, the resultsof the study show that cop- 
ing strength is an important variable to consider in 
the study of individual differences, in reading ability, 
and in the prediction of academic success. 


TABLE 5 


PEARSON r OF THREE READING TESTS 
( BEFORE DAF ) AND GRADES 


Reading Comprehension 


r -. 54 ^od 


DISCUSSION 


The overall results of this study tend to identify 
coping strength as a unique variable to consider in 
the psychology of reading as well as the psychology 
of such noncognitive variables asanxiety and self- 
concept. Individual differences were shown in the 
demonstration of coping strength as wellasthe other 
variables studied. 


One of the most interesting observations made by 
this investigator, while working with the children in 
the sample, was their exceptional motivation to con- 
tinue the exercise of reading with delayed auditory 
feedback long after their testing period was Over 
Their enthusiasm seemed to suggest that in the act ` 
of reading, the introduction of this type of distraction 
(DAF ) called fortha Vigilance that was actually en- 
joyable. It seems reasonable to assume that in ad- 
dition to the enjoyment of performing in such a m 
ner, the children found it highly motivating to be A 
volved in a feedback loop that was individualized ios 
cording to the pattern of their own performance й 


Psychologists, who st i 
y sts, чау individual differe 
uer айога оту conditions, and educators, ы CES 
pt ш the classroom, should consider not Mu 
е Sd d of coping Strength, but also the dox. 
al value o givingchildrenam le E 
Еа Pe д pte opportunity ar 
EA in y so that it may sharpen attention 3 eae 
vig aso vang thereby improve the азе, И ван 
а ре EA information proce atis RA 
; forthe psychologist 2 suns giis 
gist, the Implications of con- 


tinued resez i 
nu search into the dimensions of с 
seem highly Warranted | “ишана 


ngth 
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ABSTRACT 


The theoretical basis for the Analysis of Means Technique is discussed. In addition, a simplified work- 
ing procedure is outlined, step-by-stepforan actual problem. Thedataforthe problem are analyzed by the ana- 
lysis of means techniques, which compares differences between means instead of population variances estimates, 
The data are also analyzed by the analysis of variance technique. The conclusions reached by both techniques 
are the same. The graphical representation of the ANOM is helpful in understanding data. The ANOM technique 
is usually used in conjunction with the analysis of variance, either asan initialanaly sis of data or to augment the 


analysis of variance, 


THE THEORECTICAL basis for the Analysis of 
Means Technique is presented for the first time in 
an educational journal. In addition, a simplified 
working procedure is outlined, step-by-step for an 
actual problem. The data for the problem are ana- 
lyzed by the analysis of means technique, which com- 
pares differences between means instead of popula- 
tion variance estimates, 


The analysis of means technique is an extension 
of the Shewhart Control chart (1) which is used in 
industry for quality control. The analysis of means 
is well known in industry but it is not generally known 
among psychologists and educators. The methodis 
useful because of its simplicity and comparability 
to the analysis of variance, 


The procedure tests the null-hypothesis that k 
group means represent random samples drawn from 
a universe. If the null-hypothesis is rejected with 
risk œ , there are significant differences among the 
group means. If there are only two means, the ana- 
lysis is the student— test drawn as a graph. If k 
>2, the null- hypothesis of randomness will be re- 
jected with risk a, if at least one mean lies out- 
side the two decision lines, drawn for the graphical 
analysis of means of the experiment. 


The graphical representation of the analysis of 
means is helpful in understanding data. It provides 
а comparison of magnitudes of differences along with 
a test of statistical significance. It can pinpoint the 
presence of certain types of effects which may be 
missed by other techniques. The ANOM technique 
1s usually used in conjunction with the analysis of 
Variance, either as an initialanalysisof the data or 


E 


to augment the analysis of variance. The method is 
outlined in the following paragraphs, followed byan 
example which was analyzed using both the analysis 
of variance and analysis of means methods. 


THE ANALYSIS OF MEANS METHOD 


l. Standard Deviation Given. Let us assume a 
population with mean X" and standard deviation o', 


If we have k independent samples of n each from the 
given population and we want to consider allthe k 
means simultaneously, we compute the interval. 


EU = 
X'+ 2g 0'X 


The k means should lie within this interval to accept 
the null-hypothesis with risk œ. In order to com- 
pute Zo , let p represent the probability that one 
mean will lie between the two decision lines drawn 
at X' - Zg ox and X! + Z y о'% . Then the proba- 
bility that all k means will lie within these lines is 

p If at least one point lies outside these decision 
lines, we reject the null- hypothesis. If we consider 
the type I error as а (the risk) then Zo may be com- 
puted by the following steps: 


рк =1-@ 
k log p = log (1 - a) 
From the table of areas under the normal curve we 
obtain Zg corresponding to the value of P such that 
the area between the ordinates drawnat X'-Z,0' x 


and X' + Zqo'x isp. Clearly Z, is a function 


of k and о, so it is defined as Za (®). 
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2. No Population Standard Deviation Given. If 

k = 2 the method is similar to the Student's t-test of 
testing the significance of the difference between two 
means. When k>2 the appropriate decision linesde- 
pend upon the degrees of freedom ( v ) in the esti- 
mate 6. If k means, each based on n Observations, 
are to be analyzed simultaneously for non-random- 
ness the decision lines will be drawn at 


Х + Hy (Е, »). 


X is the grand mean (mean of means). If any one 
of the means lies outside the decision lines, the null- 
hypothesis will be rejected and this will be an indi- 
cate of non-randomness. It is clear that H is afunc- 
tion of К, v , and v. There are two sets of tables 
which locate these decision lines. One set is by Hal- 
perin, Greenhouse, Cornfield, and Talokar (2) and 
the other by Nair (3), Ott (4) uses the first set of ta- 
bles with some modifications to compute Hy (k, v) 
for some k and combinations with o = . 05 and o. 01. 


The tables are reproduced as Table 3 and 4 in theap- 
pendix ? 


Illustrative Example 


As a part of an experimental study, the College 
Entrance Examination Board’s Scholastic Aptitude 
Test (SAT) was administered to students in the Fifth 
Form (roughly equivalent to the twelfth grade in the 
U. S. А.) and the Sixth Form (roughly equivalent to 
the Junior College in the U. S. A.) in England. The 
SAT consists of a verbal section and a mathematical 
Section. The tests were administered to male and 
female students in single sexed schools in Northern 
and Southern England. The data are reported in Ta- 
ble 1 for the verbal section of SAT. Table 2shows 
the analysis of variance using a three way factorial 


design. The analysis of means will be illustrated in 
the following steps: 


The Analysis of Means 
1. For Main Effect 


If we have two or more independent variables 


817 


h; hng 
Xwi e Xwng 
карй X hart, and 
We plot the Xn Хз values on one c 
the Xyj------ Хә Values on the same chart extend- 


ed (X is the grand mean). We also determine for each 
of the different independent variables g, h, etc. the 
number of subgroups (of n each) which are pooled in 
comparing differences in the effects of eas 


and subsequently the effects of hi ----- һо›е!с. Тһеп 


we compute the standard deviations Fg + 


OR, 
etc. We find Hos and Ho; from Tables 3 and 4. We 
draw two sets of decision lines at X t Hoz? gg and 

x + Ноу xg for the g variables and similar decision 


lines for the other variables. 


TABLE 1 
THE THREE-WAY DESIGN OF SAT-V IN ENGLAND * 
North South 
Boys Girls Boys Girls 
N - 69 N - 69 N - 69 N = 69 
Sı = 31102 = 4 
sisi um Sum - 34602 Sum - 30933 Sum = 32726 

Mean = 450.7536 


Mean = 501.4783 


S. D. = 85.0753 S. D. =63. 1002 


N = 69 N = 69 


Sum = 37835 = 
sixth Sum = 38447 
Mean = 548. 3333 Mean = 557.2029 


S.D. =81.9420 S.D. = 66.0335 


Mean = 448.3043 Mean = 474, 2899 


S.D. =79.3461 S.D. = 70.9849 
N - 69 N - 69 
Sum - 38514 Sum - 36878 


Mean - 558.1739 Mean - 534 4638 


S.D. - 71.0406 


*The writer wishes to 


Ww 1 h express his a reciatior i 
providing him with the data which were collectors наз сани кещ 


а study now in Progress, 


ЖОЛ НН 


KARAS 67 


TABLE 2 
THE ANALYSIS OF VARIANCE 


s. v. D. F. SS M. S. S. E(M. S. S. ) F 
Total 551 4079284. 38 
Forms (F) 1 901776. 57 901776. 57 о2+ 276 ор? 162. 153** 
Sex (S) 1 33015. 15 33015.15 — o^. 216 og? 5. 936* 
Region (R) 1 15605. 53 15605.53  o?. 276 ср? 2. 806 
FXR 1 2416. 77 2416.77 оё. 138 ору? 0. 434 
FXS 1 72290. 84 72290.84 02+ 138 ogg? 12. 999** 
RXS 1 28336. 95 28336.95 02, 138 а. 5. 0b 
FXRXS 1 530. 16 530.16 — c^. б9орур25 0. 095 
Residuals 544 3025312. 41 5561. 24 
*p«.05 ^ 
**p«.01 о = ү 5561.24 = 74.57 


The Upper Decision Line (UDL) with (. 05) .83 « 12. 48 
= 509. 12 + 1. 39 (74. 57) .31 
Утв . 83 - 12. 48 
35 
= „12 . à —— 
ur = 6,04 . 83 + 1.83 (74.57) Уд 
У 138 
The Lower Decision Line (LDL) with (. 05) = 80.83 + 16. 48 
= 509.12 - 6.24 = 97.26 
- 502.88 LDL (. 01) = 80. 83 - 16.48 
= 64.40 
L with (. 01)= 509.12 + 1.83 (74. 57) 
The UDE w Ve (b) Form and Sex 
= 509. 12 + 8. 21 Fifth Sixth 
= 517. 33 Boys 449. 53 553. 25 


Girl 487. 88 
The LDL (.01) = 509.12 - 8.21 ш Я S45. Ей 


———————— n 
= 509.12 38. 35 -T.42 Л = 15,47 
рь t 5 = 15. 47 + 12.48 = 27, 95 
| ‚ 05) = 15. 47 - 12,48 = 9.99 
2. For Two-Factor Interaction UDL (.01)- 15. 47 + 16.43 = 31. 99 
(a) Form and Region LDL (. 01) = 15.47 - 16, 43 - - .96 
North South (c) 


Region and Sex 
Fifth 476.12 461.30 Region and Sex 


North South 


Sixth 552.77 546.32 Boys 499, 54 503. 24 


"— SE Girls 529.34 504.38 
А (Difference Mean) = 80. 83 30. 39 1.14 
UDL (. 05) = 80.83 + 1.89 3 (74. 57 LDL (o) ipio: 12.48 = pe 
i; EDE (05) «15 78. 19 4g - 34 
m UDE (01) = 15.78, je 43 42 29 
LDL (01) = 15.76 - jg 4y - 92. 19 
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TABLE 3 


L SAMPLES 
ENTAGE POINTS FOR THE STUDENTIZED MAXIMUM ABSOLUTE DEVIATE IN NORMA 
PERC 


(ХХ), (X-x 
ниш "x pus x Ox No standard deviation given 

dt ks 2 3 4 5 6 8 10 15 20 30 

5 1.82 

6 1.73 2.59 2.94 3.19 3.37 

8 1.63 2.39 2,71 2.92 3 09 33 

10 1.58 2.29 2.58 2.78 2.93 15 3.31 

15 1.51 2.16 2.42 2,60 2 74 .93 3.07 3.32 

20 1.48 2.10 2.35 2,82 2,64 283 2 96 3.18 3,33 

30 1.44 2.04 2.28 2,44 2.56 273 2 86 3.06 3.19 3.37 
40 1.48 2.00 2.25 240 252 269 2 80 3.00 3.13 3.29 
60 L4l 1.98 221 2.36 248 264 5 16 2.94 3.06 3.22 
120 1.40 1.95 2.18 2.33 2.44 “60 2.71 2.88 3.00 3.15 

1.39 1.93 2.15 2.29 2,40 655 2.65 


2.82 2.94 3.08 


k - number of means being compared 


df = degrees of freedom in estimate of error variance 


The values in 


this table are the averages of the upper and lower bounds given by Halperin, 
field and Zalokar, 


, In the Special case where we have2* factorial de- 
Sign the following procedures can be used: 


(à) Form and Region 
n6 Region 


North South 

Fifth 416. 12 461. 30 

Sixth 552, 77 546. 32 
Mean of the Fifth and North 


(FN) and Sixthand 
South (Ss) = 511.22 


Mean Fg and SN = 507, 04 
(b) Form апа Sex 
mand Sex 


Mean BF 


and GS = 497, 68 
Mean BS 


and GF = 520. 56 


e Region and Sex 


Mean BN and GS = 501, 96 
Mean BS and GN = 516, 29 


(oo oo RE 


Greenhouse, Corn- 


The two decision lines for each two 
action are identica] with that for th 


3. For 


Three- Factor Interaction 


-factor inter- 


Mean NBF, NGS, SBS, and SGF = 510, 10 
Mean NGF, NBS, SBF, andSGs = 508. 14 
NBF: The 


the three-fa i 
Íor the main effects, 


Comparin the Results of the Analysis of Variance 
and the Analysis of Means 


Table 2 shows the results of the analysis or vari- 
1 shows the results of the 


alysi 
main effec {For 18 of 


e main effects. 


—> 


TABLE 4 
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PERCENTAGE POINTS FOR THE STUDENTIZED MAXIMUM ABSOLUTE DEVIATE IN NORMAL SAMPLES 


pH 2. No standard deviation given 
2 3 4 5 6 8 10 15 20 30 40 60 
2.85 
2.62 3.74 4.21 4.53 4.78 
2.37 3.31 3.70 4.97 4.17 4,47 
2.24 3.08 3.43 3.67 3.86 4.11 4,29 
2.08 2.81 3.12 3.32 3.47 3.69 3.84 4.11 
2.01 2.70 3.98 3.17 3.30 3.50 3.63 3.87 4.02 
1.94 2.58 2.85 3.02 3.15 3.33 3.45 3.66 3.79 3.96 
1.91 2.53 2.79 2.95 3.07 3.24 3.36 3.56 3.68 3.84 3.96 
1.88 2.48 2.73 2.88 3.00 3.16 3.27 3.46 3.58 3.73 3.84 3.97 
1.85 2.43 2.67 2.82 2.93 3.09 3.20 3.37 3.48 3.62 3.72 3.86 
1.82 2.39 2.61 2.76 2.87 3.02 3.12 3.29 3.39 3.53 3.62 3.73 


Ck = number of means being compared. 
df - degrees of freedom in estimate of error variar 


The values in this table are the 
field and Zalokar. 


is helpful in understanding the first-order interac- 
tions. It is clear that the Form by Region interac- 
tion is not significant because the two lines are ap- 
proximately parallel while the two lines of the other 
two interactions are not parallel which indicates that 
their interactions are significant. 


FOOTNOTES 


1. This method was developed by Professor Ellis 
Ott, Director of the Statistics Center at 
Rutgers, the State University, New Bruns- 
wick, New Jersey. 


2. Tables 3 and 4 are reproduced by the permission 
of Professor Ott. 


3. The standard deviation estimate (14. 57) istaken 
from Table 2 or we can obtain an independ- 
ent estimate, 


псе 


average of the upper and lower bounds given by Halperin, Greenhouse, Corn- 
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BAYESIAN INFERENCE 


JON C. MARSHALL 


University of Missouri - St. Louis 


ABSTRACT 


A practical situation is used to illustrate how Bayesian inferential procedures c 
problems. In а comparison of Bayesian and classical a 


formal process involved in Bayesian procedure 


The purpose of this paper is to 
this end, an educational problem i 


THE BAYESIAN approach to statistics has been 


both damned and praised by behavioral researchers. 


Research theorists seem to present Bayesian and 
Classical Methods as two mutually exclusive re- 
search philosophies, Yet the Bayesian philosophy 
offers definite advantages to the researcher in many 
Situations, particularly in instances when a direc- 
tion of action will be decided upon at the completion 
ofthe research. In other situations, the classical 
approach seems to hold the edge. Classical meth- 
ods are more amenable to most complex problems 


involving multiple treatments and/or multivariate 
data. 


A recently published article by Marshall (1) il- 
lustrated how Bayesian decision theory could be ap- 
plied to counseling and educational problems involv- 
ing classical regression analysis. The purpose of 
this paper is to present a practical situation which 
will illustrate how Bayesian statistical theory can 
be applied to behavioral problems involving classi- 
cal hypothesis testing. 


PROBLEM 


The chairman of the mathematics department at 
Hill Top High School felt that **modern" mathemat- 
ics should be introduced into the curriculum. The 
city superintendent, who was somewhat pessimistic 
about such a change, was willing to consider the 
idea only if it could be shown that it was superior to 
the traditional method. Since the cost of new sup- 
plies and the retraining of teachers is high, afull 
Scale experimental program was impossible. But, 
it was possible to take a random sample of students 
(for expedience, say ten), give them a “modern” 
mathe matics program, and then compare them with 
students having taken a traditional program. 


CLASSICAL, APPROACH 


The classica] researcher would typically draw 
two samples of students, give one sample the ** mod- 
ern" mathematics treatment, and the other the tra- 


an be applied to educational 


pproaches to the same problem, it is shown that the 
is different from and more objective than classical procedure, 


provide a clear illustration of the use of Bayesian inferential procedures, To 


5 presented and a step-by-step Bayesian solution is provided. Itisillustrated 
that the Bayesians have come up with a cogent technique in situations where the 


choosing between two alternatives to a problem, one of which must be accepted. 


decision- maker is faced with 


ditional mathematics treatment, Each group would 
be given the same appropriate achievement test af- 
ter the treatments and then a one-tailed test (for ex- 
ample the t-test) would be used to test the null hy- 
pothesis, Но: н, - ш, = 0. The alternative would 
be H,: u, >,. Thus, the null hypothesis would be 
retained if 4, - р. «Ka and rejected if ц, - 
Ha = Ka, where Ko is the appropriate value of the 
statistic for rejection of the null hypothesis at a ley- 
el of significance. Thus, the final decision may be 
made in terms of this acceptance or rejection of the 
null hypothesis. If the null hypothesis is retained, 
the request to change the program would be denied; 
while, if it is rejected the request to change the pro- 
gram would be at least considered and possibly 
adopted. It is often argued that this is an objective 
decision. But in reality, the researcher has to make 
several subjective decisions before the final product 
is seen. First, he has to decide on the level of sig- 
nificance (v). This should be a function of the crit- 
icalness of making a correct decision, but it is often 
decided by default by rigidly placing o = ‚05, 


d x Thus, 
if done properly, the determination of œ is a subjec- 
tive decision. Secondly, what about the size of N? 


This value is determined by taking as large a sa m- 
pleas possible or by dec iding what magnitude of differ- 
ence between the means will be considered practicall 
significant. The final decision of the H 
intendent is again subjective, 
dence, he still has to decided 
p of the difference is worth t 
lecision which might be r eflected i e 
size. Thus, in at least three places inthe acne 
making process the classical researcher has jou 
ectively appraise the Situation. The Bayesian me 


ognizing this subjectivity tries to for it i 
more precise terms. зены 


School Super- 
In light of the evi- 
whether the practical- 
he cost, a Subjective 


BAYESIAN APPROACH 


Introduction 
L—rocuction 


he approach outlined here wi i 
T | зге will diff г. hi 
Classical approach on at least three ani" " p 
s: " 
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will be of little importance, 2) the criticalness of 
making a correct decision will be formally stated in 
terms of the loss function, and 3) the analysis will 
be made using the a posterior distribution rather 
than the a priori distribution. Furthermore, the 
hypotheses will be stated as the proportion of the 
**modern'' mathematics treatment group above or 
below a given point rather than the magnitude of dif- 
ference between the two means, an approach which 
is open to the classical researcher. 


Hypotheses 


The mean score on an appropriate standardized 
achievement test for those students in the traditional 
program was available from previous classes. 
Hence, the superintendent formulated the problem 
in terms of 0, the proportion of students in the 
‘‘modern” mathematics treatment group above the 
mean (M) of the traditional group. Thus, the hy- 
potheses can be stated as follows: 


Ho: 9 = .50 
Н,: 9 >.50 


Bayes's Theorem 


Before continuing with the problem, let us con- 
sider Bayes's Theorem. 


P(Ek) х P(A| Ек) 
P(E; | A) =— ; 


J 
E P(Ei)*P(A|Ei) 
i- 


1 


where P(A) 4 0; i» 1, 2, ..., j; Ej represents the 
various values of 9; and A is a particular exper i- 
mental value that has been observed. Inother words, 


probability of [à prior! probability Probability of obtaining 
having a specifie of the state of nature} * | the experimental result 
state of nature given the state of nature 
given an experi- 

mental result 


xw | Aptian proba- Probability of obtaining | 
2, | bility af the * | the experimental result 
С Listate of nature given the state of nature 
for all 

possible 

states of 


nature 


The state of nature in this case refers to the actual 
proportion of a population of **modern'' mathematics 
Students who would score above the mean score for the 
"traditional" students (previously notedas 9). From 
the Bayesian viewpoint, the purpose of an ex peri- 
ment is to modify one's opinion rather thandiscover 
Something new. That is, there is a non-zero prob- 
ability of occurance which can be personally as- 
signed to any event. The purpose of an experiment 
is to modify this personal probability in light of new 
evidence, Therefore the а priori probabilities cor- 
responding to the various states of nature [P (E1)] 
have to be determined as an initial stage in the ex- 
PERI process. The experimental probability 
P(A| Ej )] is simply the binomial probability of ob- 


taining the experimental › 
value is Ej. в ntal value of A if the population 


A Priori Distribution 


Knowing very little about the ас! istributi 
tual distributio; 

of scores, the superintendent decided that it ag 

just as likely that 50 percent of the sample would 


score above the mean of the traditional group as it 
was that 45 percent or 55 percent would score above 
the mean. It was further argued by the superintend- 
ent that more than 60 percent of a **modern'' math- 
ematics population would not score above the mean 
of the traditional group or the value of “modern 
mathematics would be so obvious as to warrant any 
other consideration impossible and that if less than 
40 percent would score above the mean, the ineffec- 
tiveness of the program would be so obvious as to 
have no supporters. Thus, it was decided that the 
a priori distribution should be rectangular with a 
total range of 21 points, with end points at . 40 and 


.60. Therefore, it can be determined that the P(Ei 
1/21 2.048, where E, -.40, Е, =.41 er Baa = 
. 60 (See Figure 1). 


It should be noted that the distribution with which 
we are working is binomial. That is, there are only 
two possible observations that can be made. The 
first is if the observed proportion of scores (A/n) 
above the value M is >. 50 and the second is if the 
observed porportion of scores above the value M is 
= .50. Thus, the probability function is 


( .48 for A/n >. 50 
P(A =a) = 


T .82 for A/n =. 50 


That is, the a priori probability of A/n >. 50 is . 48. 
This can be determined by adding the a priori prob- 
abilities [ P(Ej)] for Ej =.51, .52, ..., . 60. 


Loss Ratio 


Before continuing, the superintendent had to de- 
cide what probabilities that 2 >. 50 (H,) would be 
sufficient for him to take action. This decision is 
dependent upon the losses of utility. That is, what 
would be his loss if he decided 4 = .50 when 9 >. 50, 
L(25), and what would be his loss if he decided 
9 >.50 when 8 = .50, L(0,). These loss values аге 
determined using both objective and subjective cri- 
teria such as the amount of expenditures, loss to 
students taught using an inferior method, lossof rep- 
utation of the superintendent for makinga bad decis- 
ion, and loss of time spent. Thelosses might be 
figured as follows: 

I. Possible loss when deciding in favor of ‘‘mod- 
ern" mathematics (H,) 
A. Expenditures 
1. New ‘‘modern’’ materials — 


$5,000 
2. Retraining of teachers — 
$5,000 10 units 
B. Students 5 units 
C. Reputation 2 units 
D. Time 5 units 
1(9,) Total 22 units 


. Possible loss when deciding in favor of tradi- 
ditional mathematics (Ho) 
A. Expenditures – **Up-to-date'" 


material ~ $2,000 2 units 
B. Students 5 units 
C. Reputation 2 units 
D. Time 2 units 


L (90) Total 11 units 


The relative weight of the importance of each loss 


"v 
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is strictly personal to the decision-maker. It could 
be the importance of each type of error as he sees 
it or it could be the importance of each as a com- 
mittee sees it. But, it is a personal weighting of 
each factor. For purposes of simplicity let us as- 
sume that the weighting was personal to the super- 
intendent. Then, as he saw it, the loss of $2,000 
was as important as the loss of reputation he would 
incur if he was wrong. The loss to the students 
would be 2 1/2 times greater thanhis personal loss. 
There was a time loss in both cases, but the time 
loss when deciding in favor of the change was much 
greater than the time loss when deciding against the 
change. After expressing each segment in relative 
units, the relative loss for each error can be deter- 
mined, The relative losses are thus L(@ 9) = 11 and 
L (0,)- 22. 


The possible loss when deciding in favor of H, 
is twice as great as when deciding in favor of Ho. 
This is expressed as a decimal fraction in the loss 
ratio. The loss ratio can be defined as: 


This loss ratio will play a key role in the final de- 
cision. 


A Posteriori Distribution 


Suppose that the sample of ten students was 
treated using ‘‘modern’’ mathematics. What is the 
probability of a state of nature existing (Ej) if a cer- 
tain experimental value (A) is observed (i. e., 
Р(Е; |А) )? This is nothing more than calculating 
the probability for each Ej using Bayes’s Theorem. 
But, to do this the probability for A given Ej (i.e., 
P(A |Ei ) ) is needed. Since the prior distributionis 


чч==ш>»шош'ч 


rectangular and thus P(E,) = Р(Е,) =... = P(Ei), 
.048, the P(Ei) term can be factored out of 
both the numerator and denominator of Bayes's the- 
orem, or 


Р(Ек) P(A| Ек) P(Ei) P(A| Ek) Р(А|Ек) 
P(Ek| A)- = = » 
EP(Ei) P(A| Ei) P(Ei) EP(A| Ei) EP(A |Ei) 


The prior conditional probabilities P(A | Ei) for A = 


2, 3, ..., 8 given Ej are shown in Table 1along 
with the accumulated posterior probabilities 
P(Ei|A) for Ej given A = 2, 3, ..., 8. The graphs 


of these accumulated posterior probabilitiesare 
also shown in Figure 1. The prior conditional prob- 
abilities (Pal Ej) ] are nothing more thanthose 
probabilities determined using the binomial distri- 
bution where n = 10, x = A, and p = Е}. 


P(A |E) = (4) (E)^ (1 - ^^. 


The posterior probabilities are those determined 
through the use of Bayes's theorem. For example, 


P(A = 2|Ej =. 40) = ( 19) (40) (60) = .121 апа 


21 


У Р(А = 2 ЇЕ) = 1,092. Therefore, the posterior 
iz1 


probability for Ej =. 40 is 
P(A = 2|Ej =.40) .121 
21 j 


ZP(A-2|Ei) 
i=l 


P(E; = . 40| A = 2) = 


=, 211, 
1, 092 


Table 2 gives the posterior probabilities that 
0 = .50 for A = 2, 3, ..., 8 and the corresponding 


STATE OF NATURE (Ej) 


FIGURE 1. Rectangular Prior Probabilii 
Distributions for A - 2, A - 4, А = 6, and 


Distribution, and 


е Posterior Probability 
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Odds ratios. The odds ratio сап be defined as the 
numerical chance that 9 >. 50 after observing that 
A = ак. Thus, the odds ratio is 


Р(8,|ак) 1-P(]ag) | 


pai) =— 55 
(ay P| a) P(95| ay) 


For example, the odds ratio of 9 >. 50 when A = 2 is 


.792 — .208 2 1 


.792  .199 8 4 


5(2) 


At this point, the decision (D) can be made objec- 
tively. 


D = (odds ratio) * (loss ratio). 


If D <1, the superintendent will decide that 35 .50 
and deny the request to adopt a **modern" mathe- 
matics program. If D >1, he will decide that 
9 7.50 and grant the request to adopt a “modern” 
mathematics program. If D = 1, he would reserve 
his decision until a few more cases were observed, 
Table 3 gives the D values for the various values of 
A. The D values in Table 3 indicate that if A>7 the 
superintendent would decide in favor of the proposed 
"modern" mathematics program, but if А < 7 he 
would decide against it. 


DISCUSSION 


In the illustrated problem, the Bayesian re- 
searcher would require A > 7 before rejecting Но іп 
favor of H,. When using Bin (10, . 48) a priori dis- 
tribution with о = . 05, the classical re searcher 
would require A = 8 before making the same deci- 
sion. Thus, by not formally determining the loss 
function nor taking into consideration the posterior 
distribution and sticking close to the . 05 level of sig- 
nificance, the classical researcher would be overly 
strict. In order to make the same decision as the 
Bayesian (who is taking these things into considera- 
tion) the classical researcher would have to set 
8 2.142, an unlikely possiblity, In actuality, be- 
cause of the seriousness of the action, the classical 
researcher might even require 9 = , Olandthus A 29 
for acceptance of H,. 


The Bayesian approach to statistical inference of- 
fers some definite advantages to the researcherover 
the classical approach. The formerapproachforces 
the researcher to determine the losses to be in- 
curred if he makes a decision and is wrong. It al- 
lows the researcher to base his decision on the 
latest information (posterior distribution) while tak- 
ing into consideration prior information without be- 
ing restricted by a priori assumptions regarding the 
Shape of the posterior distribution, such as the as- 
sumption of normality necessary for most classical 
designs. Furthermore, the final decision is 
objectively based on the product of the odds ratio 
and the loss ratio, Even though the Bayesian ap- 
proach does have these advantages, it should not be 


TABLE 2 


POSTERIOR PROBABILITIES FOR 9 
CORRESPONDING TO VARIOUS VALUES OF A 


A = ак 
[Observed P(@ >. 50) Odds Ratio 
Frequency] P (4 = . 50) [ 1- р = .50)] (9 >. 50) 


2 .792 .208 1:4 
5 а .283 3:7 
4 .624 -576 2:3 
5 .525 .475 ма 
6 .425 ‚575 3:2 
7 .329 .671 1:8 
8 . 248 - 152 3:1 


P( 9 = ‚50) = P(@ = .50| A) = Accumulated Posteri- 
or Probabilities in i = 11 row in Table 1 


TABLE 3 


D VALUES FOR VARIOUS VALUES OF A 


A =a, L(?) X 8 (ap) =D 

2 (1/2) (1/4) = .125 
3 (1/2) (3/7) = . 214 
4 (1/2) (2/3) = . 333 
5 (1/2) (1/1) = . 500 
6 (1/2) (3/2) = . 750 
T (1/2) (7/3) = 1, 170 
8 (1/2) (3/1) = 1. 500 


considered a cure-all for sta 
Situations where the decis 
choosing between two alternativ 


Situa- 


the Purpose ig 
is 
to integrate into 


not an action decision to 
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COLLEGE STUDENT RATINGS OF INSTRUCTORS 


NICHOLAS F. RAYDER 
Michigan State University 


ABSTRACT 


In a great many of our institutions of higher learning, 
for students to rate instructors. Often, the summarized 


dent ratings over five thousand college students rated their instructors on each of three criteria. These rat- 
ings were then correlated with selected student characteristics and instructor characteristics. The technique 
used was multiple linear regression. Generally, mean student ratings of instructors were more related to in- 
structor characteristics than to such student characteristics as age, grade level, GPA , or course grades. 


FRED M. HECHINGER, education editor of the City College of New York, a joint student-faculty 
New York Times has written: committee was established to develop ina truly col- 
laborative fashion some type of course evaluation. 
A program evolved consisting of the distribution of 
course-instructor questionnaires to instructors on 
a voluntary basis. The instructor distributed the 
Scales, collected them, and was free to use them 
in any way he wished. He could send them to the 
office of the Chairman of the Department of Student 
Services where a confidential report would be pre- 
He further stat pared and returned to him. Of 400 instructors dent 
educational policy n dE iid seek to sit on questing to participate, only 144 sent the collected 
Sibility that faculty БЕЛА сы жоны De Ет n а анааан 


Students have persuaded the edu- 
cational leadership that the student's 
voice on campus matters deserves to 
be heard and that their complaints 
about inadequate teaching and a cur- 
riculum out of step with contempor- 
ary society are often justified (2), 


These student com 
rest on campuses thr. 
at large universities 


plaints are dramatized by un- 
oughout the Country. Students 
complain about being disturbed 
us Somewhat threatened by the dehumanization and 
e loss of identity they perceive in the multiver- 


Sity. Many are deepl; 

y concerned about t 
of our system of higher educ. ejiis 
most noteworthy in the impr 


Other faculty groups are responding to students 
who want to have a greater role intheaffairs of their 
institution. In 1965, Cornell University appointed 
a Committee on the Quality of Undergraduate Instruc- 
ч tion chaired by Dr. Alfred E. Kahn (3). Although 
ation and want change, appointed by the executive branch of the univer- 
ovement of instruction. sity, the members worked closely with the student 
body to study the academic aspects of undergraduate 


Freder 
liams Coll education at Cornell. The comprehensive report 


, hing new in the desire of i ded the followi 
Ane ед about change and reform on MM таш чааныр шен тен 
mpuses. He states: « 3 
the college is th 5: “For a few years 
ve it serveg Ub ECL. and they (students) will l. Grossly inadequate teaching occurs in more 
there are those Er hel) Gon ik wish it, unless instances than is tolerable. 
haps wiser choice (2), О some other, per- 


trators are helping by eee хану and adminis- 2. 


ў There сап be no doubt that student dissatis- 
ganizations wanting to i 


í faction wi i i 
mprove instruction, At the ae Cae ee instruction has 
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3. There is only one ultimate determinant of the 
quality of undergraduate instruction and only 
one ultimate source of its improvement—the 
individual teacher himself. 


The committee recommended unanimously that 
“teachers obtain anonymous student evaluations of 
their courses" and, with one member dissenting, 
“that the university give Student Government finan- 
cial and technical assistance in preparing and pub- 
lishing competent and responsible campus-wide stu- 
dent evaluations of courses” (3). 


A similar committee at Michigan State Univer- 
sity, The Committee on Undergraduate Education, 
was appointed in 1967 by the school's president to 
reexamine the undergraduate program and to pro- 
pose desirable changes. In a section of their final 
report concerning student evaluation of teaching, the 
committee stated that ‘‘an instructor should know 
when students do not approve of a particular tech- 
nique or approach (1). The committee recommend- 
ed that a special all-university office be given the 
responsibility and adequate support: to revise or 
develop new Student Instructional Rating Forms; to 
collect and disseminate normative data relative to 
these forms; and to analyze the results and prepare 
summary reports." They further recommended that 
‘Faculty members be encouraged to have these sum- 
mary reports sent to their department chairman so 
that the results can be involved in decisions pertain- 
ing to them. ” 


The number of colleges and universities using 
Some method of student evaluation is increasing rap- 
idly and not without some adverse consequences, Ob- 
viously, there are many dangers when student rat- 
ing of faculty performance тау be a factor ina 


teacher’s professional standing, possibly affecting 
his promotion. 


If studies are not methodologically sound, popu- 
larity might rate higher than intellectual rigor; or 
political views might influence decisions that ought 
to be arrived at by a process of free inquiry. 


Other questions concerning the validity of student 
ratings are equally critical, and this report su m- 
marizes a study designed to examine several stu- 
dent and instructor characteristics to see what re- 
lationship each had with student ratings of their in- 
Structors. 


The research was carried out within the School 
of Education at Colorado State College (Greeley, 
Colorado) during the 1967 winter quarter. The spe- 
cific instructor population was identified as all fac- 
ulty within the School of Education who were teach- 
ing courses during winter quarter. The Student pop- 


Ulation was all those who were enrolled inthose 
Courses, 


PROBLEM 
. No attempt was made to identify effective and 
epee instructors. Instead, the study concen- 


Upon variables believed to have some rela- 


tionship to student ratings. Therefore, the major 
questions studied were: 
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1. Does student sex, age, major, level of edu- 
cation, GPA, or course grades previously 
received from the instructor being rated have 
any relationship to ratings of instructors? 


2. Are instructors who differ in sex, age, faculty 
rank, degree held, major area, or length 
of teaching experience rated differently by 
students? 


A further question concerned the accuracy of 
students’ reports of their GPA's. This was done 
to check the validity of student information. If large 
discrepancies existed between students’ reported 
GPA and their actual GPA (determined from col- 
lege records) the remaining information provided 
by students would be suspect. 


METHODOLOGY 


To collect the necessary data, a 1-page Student 
Scale was constructed. The scale asked students 
to provide information about themselves and to rate 
their instructor on each of three 7-point continuums, 
The referents on each continuum were identified by 
Ryans as being important teacher traits or patterns 
of behavior (4). These were: 


I. aloof,egocentric, vs. friendly, understanding, 
restricted behavior behavior 


П. evading, unplanned, vs. responsible, system- 


slipshod behavior atic, businesslike be- 
havior 
II. dull, routine vs. stimulating, imagina- 
behavior tive enthusiastic be- 
havior 


, A short Instructor Characteristics Scale was de- 
signed to obtain from faculty such information as 


Sex, age, rank, degree held, major area, and 1 
of teaching experience, ЕШ 


Next, a group of doctoral students 


were i 
to administer the scales, At the insi ned 


tructors' con- 


€, the Instructo. 
was administered, á 


> performed t 
the questions. One statisticaltechn ae 


FINDINGS 
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TABLE 1 


CORRELATIONS AND RSQ’s BETWEEN CRITERIA AND STUDENT CHARACTERISTICS 


Student Characteristic (s) 


(Sex, Age, Level, 


Rating Continuum Reported GPA Previous Grade (s) Major) 
N r N r N RSQ 
D 4,032 -.03 753 .14 4, 285 . 006 
п 4,032 -.02 753 .01 4, 285 .015 
IH 4,032 .03 753 .18 4,285 .019 


l. Students remembered and accurately report- 
edtheir GPA's, The correlation for one 
hundred randomly selected student-reported 
GPA's and their actual GPA's was . 96. 


2. Student ratings of instructors were not sub- 
stantially related to student's sex, age,grade 
level, major area, GPA or grade(s) previous- 


ly received from the instructor they rated 
(Table 1). 


As indicated in Table 1, the highest correlation 
(. 18) was obtained between students previous 
grade(s) earned from the instructor they were 
rating and their rating of that same instructor 
on continuum III. Also, there was no relation- 
Ship, reflected by correlations of -. 03, -.02, 
and . 03 between students GPA and their ratings. 


When the student characteristics of Sex, age, 
grade level, and major were used collectively 


TABLE 2 


PREDICTED INSTRUCTOR RAT 
HYPOTHETICAL UNLIKE STUDENTS 


INGS FROM STUDENT 


to predict ratings, the RSQ's were all less than 
.02. To demonstrate the low predictability of 
this set of characteristics, the regression weights 
associated with each characteristic were usedto 
predict how two hypothetical students would 
have rated an instructor. These predictions are 
found in Table 2, 


Based on the regression equation, in which sex, 


age, level of education, and major area were 
used, Student No. 1, a 35-year-old male grad- 
uate student majoring in educational adminstra- 
tion would be predicted to rate an instructor 3, 5 
on continuum I, 2. 7 on continuum II, and 2. 7 on 
continuum III. Another student (No. 2), ап 18- 


year-old female majoring in elementary educa- 
tion was predicted to rate an instructor 3. 3, 2.8, 
and 2. 3onthethree respective continuums. These 
small differences in predicted ratings (0. 2, 0. 1, 
and 0. 4) for quite dissimilar Students demon- 
strated the low relationship between this group 


of student characteristics and student ratings of 
instructors, 


CHARACTERISTICS FOR TWO 


Characteristics of Student 


Predicted Rat- 


ing on 
Continuum 
Student Sex Age Level of Education Major I п ш 
1 Male 35 Graduate Educ. Admin. 3.5 2.7 2.7 
2 Female 18 Freshman Elem. Educ. 3.3 2.8 2.3 
c 


Difference 0.2 0.1 0.4 
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TABLE 3 


RSQ's FOR INSTRUCTOR CHARACTERISTICS AS PREDICTORS OF STUDENT MEAN RATINGS 
ON CRITERIA FOR EIGHTY-SEVEN INSTRUCTORS 


Model Instructor Variables Variables Deleted Continuum Predicted RSQ Value 
d Sex, Age, Rank, Degree, None I 215 
Dept. Years Exp. п E 


2 Age, Rank, Degree, Dept., Sex I 2415 
Years Experience п cst 


3 Sex, Rank, Degree, Dept. , Age I .15 
Years Experience II .18 


4 Sex, Age, Degree, Dept., Rank I ‚15 
Years Experience п 


5 Sex, Age, Rank, Dept., Degree I .15 
Years Experience II .27 


6 Sex, Age, Rank, Degree, Dept. I 
Years Experience 


7 Sex, Age, Rank, Degree 


Years Teach- I . 14 
Department 


ing Experience п .25 


8 Sex, Dept. 


Age, Rank, I ait 
Degree, Years п .13 
Experience ш 13 


3. Generally, mean student ratings of instructors 


i i n classificati i 
showed more relationship to instructor charac- equation (Model пае мав наб y om the 
isti . SQ va Я 
ораи um I dropped from . 15 to , 05. pepe e a 
ы ч 
For the instructor characteristics of sex, age, E - 27 to . 14; and for continuum Ilfrom 
degree held, faculty rank, and number of years И о . 08, 


experience, collectively, the RSQ was . 27 for 


Continuum II (‘‘evading"’ vs. businesslike’’). Mean student ratings accord 


is means that 27 percent of the variability reflected this reduction i sue to major areas 
А b í ucti ». 
E ratings on continuum II is related to this set means calculated for ean а pp, A On example, 
of instructor characteristics. “aloof vs, friendly“ continuum теті ^ the 
to 5.8, on the ‘un, nged from 4 0 
, planned v. "ees 
4. The greatest reduction in the index of predicta- from 3. 9 to 5, 5. Systematic Scale 


bility (RSQ) was observed when the **department" 


Ing" scale from 4.1 to 5. 9 "dull vs. stimulat- 
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TABLE 4 
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INTERCORRELATION COEFFICIENTS BETWEEN INSTRUCTOR CHARACTERISTICS AND 


THEIR AVERAGE RATING BY STUDENTS 


Variable 


1 2 3 4 5 6 
(1) Continuum I 
(2) Continuum II .47 
(3) Continuum Ш ^i . 67 
(4) Age -.17 -.15 -.25 
(8) Faculty Rank 2.15 15 -.06 -51 
(6) Highest Degree Held -.16 219: -.05 .91 ‚19 
(7) Years Teaching Experience -.18 .00 =, 27 . 86 .54 „81 
5. The correlations of the three student ratingcri- b. Ryans (4), using factor analysis, identified 
teria with instructor age, faculty rank, highest three patterns of teacher behavior in the 
degree held, and years of teaching experience Characteristics of Teachers study. How- 
yielded twelve correlations varying from -. 27 ever, in the present study, when each of 
to + . 15 (Table 4). these patterns was placed on a rating con- 
tinuum and administered to college students, 
Nine of these were negative, suggesting that the intercorrelations among ratings were 
older, more educated and more experienced high (Table 4). This suggested that the traits 
instructors were rated lower than younger and Ryans identified as being independent do over- 
less experienced instructors. Further, be- lap. Therefore, a total student rating score 
cause mean student ratings were used as an of an instructor could have been obtained by 
indicator of each student's rating, correlations adding the student ratings made on each con- 
were attenuated. If each student's actual rat- tinuum. 
ing scores were used, correlations would have е 
been higher and the inverse relationship mag- с. The study gave a large portion of the student 
nified. body a chance to rate their instructors. Of- 
ten, when such a route is not provided or 
such a desire is thwarted, the upshot may 
DISCUSSION be a poorly organized student-directed eval- 
uation resulting in a faculty ‘‘blurb’’ sheet. 
To involve students in any system of faculty 
evaluation seems perilous. Yet it is students who d. After student leaders in charge of develop- 
have the most contact with instructors. To ignore ing an instructor evaluation program talked 
them seems even more perilous. The results of with this investigator, they appreciated the 
this study confirmed that student ratings are not in- time, sophistication, and financing needed 
fluenced even by variables such as grades previously to carry out a methodologically sound stu- 
earned from the instructor being rated. Since stu- dent evaluation. And, after examining the 
dent ratings were found to be relatively unbiased, complexity of the issues, these student lead- 
more confidence could be placed on the results of ers agreed that some degree of confident - 
how certain instructor characteristics were reflect- iality should be maintained and that student 
ed in students’ ratings. As was mentioned, these publications on course evaluations should 
results suggested that ratings were different from contain more objective information such as 
various departments within the School of Education. the number and type of exams given. 
Further, the younger instruc! i - 
Mer хее and fewer Sears of (Guy ean 
onsistently rat “fri 
ings Ан ed more ''friendly, understand- SUMMARY 


t ponsible, businesslike; stimu- 
lating, imaginative, and enthusiastic. "n i 
Probably as important as the ific findi 
ific findings 
the investigator made sev BEEN tdi 
eral observations i 
not lend to statistical analysis: TEM 


а. Tue multiple regression approach used in 

E is study was effective in analyzing the 

Mo ena mih Certain student and instruc- 
racteristies w i 

dent sates ere reflected in stu- 


As a consequence of the study reported here, we 
now have more information on the nature of college 
students ratings of their instructors, at least at Col- 
orado State College. We have shown that some in- 
formation reported by students is accurate. We have 
shown that student ratings are not influenced by such 
variables as previous grades received from the in- 
structor they rated. And we have shown that instruc- 


tors who differ on certain characteristics are rated 
differently by their students. 
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Still other equally important issues remain un- 
answered. We need to know what student charac- 
teristics, if any, do influence their rating of in- 
Structors. We need to know if a parallel study per- 
formed within another academic area or at another 
institution will yield similiar results. And, weneed 
to know if there are other instructor characteristics 
that better reflect student ratings. 


Yet, in terms of students rating their instructors 
the question: ‘‘to rate or not to rate’’ can best be 
answered by citing a quotation attributed to Aris- 
totle: ‘‘You get a better notion of the merits of the 
dinner from the dinner guests than you do from the 
cook. "’ 
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PREDICTION OF COMMITMENT TO THE 
CHEMICAL BOND APPROACH TO TEACHING 


CYNTHIA V. L. WARD! 
Rhode Island Junior College 


ABSTRACT 


In general terms, the problem under investigation is that of determining whether selected teacher character- 
istics can be used as predictors for the commitment of teachers to a certain approach to the teachin g of high 
school chemistry (Chemical Bond Approach). Teachers attending a summer institute were initially surveyed to 
determine these characteristics; included in this group were expectations of role related groups, innovativeness 
characteristics, and personal characteristics. At the completion of the institute, the participants were tested 
regarding their attitudes toward and their willingness to use the approach. Employing the stepwise regression 
technique of analysis, two characteristics (perceived disposition of reference group and innovativeness) were 
found as strong predictors. Pre-institute characteristics were also shown to account for a significant portion 


of the commitment of teachers to the approach. 


THE PROBLEM 


Since the inception in 1953 of National Scie nce 
Foundation summer institutes, hundreds of thousands 
of high school teachers have been exposed to new de- 
velopments in the teaching of science and mathemat- 
ics under the auspices of these programs. Despite 
the size of this effort it would be difficult to gauge 
the impact that these summer institutes have hadon 
the evolution of high school science teaching in the 
United States. The deter mining factor controlling 
the change attributable to these institutes is the de- 
Cision made by the individual participant on his re- 
turn from an institute as to the modifications he may 
make in his own teaching in light of the summer's 
experience, 


rea in thi i i i 

аш this Choice-situation, and whether certain 

P nones of the teachers prior to their involye- 

Eus е institute could be used to predict the 
lolce that they would make, 


and 
to further е reference gro 


Onstruct the theos theory (8) 


A were used 
etical basis, 


METHOD 


The decision makers in this experimental setting 
were the high school chemistry teachers attending a 
CBA institute sponsored by the National Science 
Foundation in the Summer of 1965. The pre-insti- 
tute characteristics of the teachers were considered 
to be possible determinants in affecting the decis ion 
in this situation, Various degrees of commitment 
on the part of the teachers to the CBA method of 
teaching high school chemistry at the completion of 
the institute were selected as the alternatives of the 
decision process (see Figure 1). 


Due to the fact that the CBA institute provided 
only the vehicle for the decision, it was not neces- 
sary to explore the course offering in detail, It is 
sufficient to say that the indications were that teach- 
ers completing the course in the past had mixed feel- 
ings as to the applications of this approach in their 
own teaching, thus the possible alternatives of this 
decision situation were expected to vary alongacon- 
tinuum ranging from total rejection to total commit- 
ment to the CBA method. This research project 
was not designed to include evaluations of the CBA 
institute attended by the sample or the CBA method 
of teaching high school chemistry. Ergo, any infer- 
ences concerning such evaluations based on the data 
collected in this study are unwarranted. 


Determinants which were investigated as possible 
predictors for the choice- situation were divisible in- 
to three distinct sets: A. the expectations of the 
groups in role relationship to the high school chem- 
istry teacher; B. the innovativeness of the teacher 
and his teaching environment; and C. personalchar- 
acteristics ofthe teacher. The listing below pre- 
Sents the selected determinants hypothesized to be 
indices of commitment to the CBA method. In hy- 
re ne these factors servedas independent 


A. Perceived Expectations of Role Related Groups 
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. Membership group 
. Other direct contact group 
. Reference group 
. Authority group 
. Composite group 
B. Factors Related to Innovativeness 
1. Innovativeness of school administration 
2. Pre-institute use and attitude toward CBA 
materials 
3. Pre-institute use and attitude toward CHEM 
Study materials 
4. Degree of professional activity (reading and 
attendance at meetings) 
C. Personal Characteristics 
Sex 
Age 
Credit hours earned in chemistry 
Credit hours earned in physical chemistry 
. State in which teacher was employed 


owe 


op сого 


The criterion or dependent variable, considered 
to be commitment to CBA, was conceptualized as be- 
ing composed of an attitudinal component and a be- 
havioral readiness component. To assess the atti- 
tude toward the CBA method of teaching high school 
chemistry, the individual teachers were asked to re- 
spond to a series of evaluative statements concern- 
ing the method. Behavioral readiness was gauged 
by the response made by the teacher when presented 
with a simulated situation in which a choice concern- 
ing the use of the CBA method was necessitated. A 
high degree of correlation was found, as expected, 
between the attitude and behavioral readiness com- 
ponents of the commitment variable. 


PROCEDURES 


The Sample 


The sample was composed of 45 chemistry teach- 
ers from American public and independent schools. 
The sample was not assumed to be representative 
of the universe of high school chemistry teachers, 
or of high school chemistry teachers attending CBA 
institutes. Consequently, care should be exercised 
in generalizing from the results, though the results 
should provide indicators as to the reactions of oth- 
er groups in similar choice-situations. 


Development of the Research Instruments 


Two written questionnaires were selected as the 
most effective and efficient means of collecting the 
data for this project. Questionnaire I ¢ ontained 
items seeking to determine: 1) the background vari- 
ables used to describe the sample, and 2) the pre- 
institute characteristics of the participants. Ques- 
tionnaire II contained items aimed at distinguishing 
the extent of the teacher's commitment to the CBA 
method at the completion of the institute, 


Both questionnaires were subjected to a pre-test 
conducted by correspondence, employing the 1964 
Participants of the same CBA institute. The results 
of the pre-test were used to revise the questions in 


an effort to reduce ambiguities and possible misin- 
terpretations, 


Questionnaire 1: As indices to the first group of 


independent variables, the teachers were asked to 
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respond to a series of items which sought to dis- 
cover their perceptions of the dispositions of others 
to the CBA course. Five major groups (high school 
chemistry teachers, high school science supervisors, 
high school principals, college chemistry teachers, 
and science educators) were selected as groups 
which the teachers would anticipate to hold expecta- 
tions for the use of CBA materials inthe high 
Schools. A series of items was included in the first 
questionnaire to determine which of these groups 
served as reference groups (groups to which the in- 
dividual aspires to attain or retain membership)? 
and authority groups (groups which are recognized 
by the individual as having special knowledge in the 
area) for the individual teacher. Using these items, 
the dispositions toward CBA perceived by the teach- 
ers for their membership group, other direct con- 
tact group, reference group, authority groups, and 
the composite groups were determined. Other items 
were included on this questionnaire to measure the 
innovativeness factors and the selected personal 
characteristics of the teacher. Some sample ques- 
tions from Questionnaire I are presented in Example 
l. A set of arbitrary scales was designed to assign 
scores to the subjects on each of the independent 
variables. 


Questionnaire П: The set of items devised to 
measure commitment to CB A was differentiated 
for three general types of high school chemistry stu- 
dents (college-bound science, college-bound non- 
Science, and terminal high school), because of the 
anticipated difficulty of treating high school chem- 
istry students as a homogeneous group. With the 
use of the two components (attitude and behavioral 
readiness) and the three groups, six scales were ob- 
tained to represent the commitment variable, In 
order to retain information, these scales were not 
reduced to one, though they were highly correlated, 


In the final analyses 13 items were used to meas- 
ure the attitudinal component ? and eight items were 
used for the behavioral readiness component, See 
Example 2 for samples of these items. 


The procedure of principal-component D i 
c analys 
was used to establish that the items for each of the 
two components for the three different groups of stu- 
dents contributed to unidimensional scales, For the 


accounted for at 


AT score tec 
of the subjects а 


the dependent variable, The T sc 


hnique was utilized 
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EXAMPLE 1 


SELECTED ITEMS FROM QUESTIONNAIRE I 


Assume that you are qualified for all of the following positions. Which one would be your first choice for a 


position at the present time? 
a) secondary school 
chemistry teacher 
d) college teacher of 
Science education teacher 


b) secondary school 
Science supervisor 
e) college chemistry 


с) secondary school administrator 


From which one group would you most value a compliment or a word of praise on your teaching of high school 


chemistry? 


a) associate chemistry teachers b) science Supervisor 
e) college chemistry teachers 


d) science educators 


c) high school principal 


From which one of the following groups would you most likely seek advice in dealing with a curriculum problem 


for your high school chemistry class? 


a) associate chemistry teachers b) science supervisor 
€) college chemistry teachers 


d) science educators 


c) high school principal 


What opinions do you think that most high school chemistry teachers hold for the use of CBA materials in the 


high school? 
a) favorable 


b) neutral 
d) no impressions 


c) unfavorable 


Which of the following statements best expresses the reaction of the principal and assistant principals in the 
school in which you teach toward developments in curriculum? 


a) Desire the school to be at 
the forefront in using and 
testing experimental 
curriculum. 

d) Are very reluctant to make 
changes in curriculum. 


b) Are willing to accept cur- 
riculum developments if 
they have proved successful 
in trials in other schools. 


C) Are receptive to developments 
that are already in use in the 
majority of schools. 


What use, if any, have you already made of the CBA materials in your own teaching? 


a) none b) slight use 
d) extensive use 


c) moderate use 


e) taught entire course 


Indioate the extent to which you are familiar with the CBA test. 


a) have read the text 


b) have scanned the text 


c) have not seen the text 


ANALYSIS 


termine which of 
nificant predictor 


A cut-off value for thi 


e F ratio was 
that only those independent as Selectedso 


А іпдер ent variables whose beta 
DS Significant, as determined by use of a 
d$. Es а. 10 1еуе1 of Significance, were included 
the à етене equation. The data obtained from 
nalyses appear in tabular form in Table 2, 


RESULTS 


the perceived disposition of the teacher's reference 
group and innovativeness. Each of these factors 
was in evidence in all but the behavioral readiness 

component — college-bound science student group, 
where the reference group disposition factor did not 
appear. 


The innovativeness factor was redefined after the 
analyses as a composition of three of the sub-groups 
originally conceived as contributing to this вонь 
These variables (including: 1. the perceived inno 
vativeness of the home administration in curriculum 
matters; 2. prior use of CBA materi als; and 3. 
prior use of CHEM Study materials) were found to 
exhibit a duplication effect because of the high posi- 
tive correlation between them. Thus, the removal 
of one of these variables from the correlation ma- 
trix resulted in a lowering of the partial correlation 
of those variables in the group remaining in the ma- 
trix. Consequently, these variables were consid- 
егей as a group and were labeled for future refer- 
ence as the innovativeness variable. 


Other independent variables which were signifi- 
cant predictors for one or two of the six measures 
of commitment were: 1. whether the teacher was 
employed in New York State; 2. the number of under- 
graduate and graduate credit hours in chemistry 
earned by the teacher; and 3. the amount of profes- 
sional reading and number of professional meetings 
attended by the teacher. 


Teaching in the State of New York wasa variable 
which was represented twice on the attitudinal com- 
ponent as a predictor. If a teacher was employed in 
New York he was less likely to be strongly commit- 
tedtothe CBA method than were teachers from oth- 
er states, Comments by institute participants rein- 
forced the suspicion that teachers from New York 
felt restricted in the extent to which they could im- 
provise upon the syllabus set for the New York Re- 
gents Examination. These same individuals ex- 
pressed the hope that the revised syllabus, which 
was scheduled for introduction in 1968, will include 
certain topics found in the CBA presentation. 


The last two variables proved to be negative pre- 
dictors; thus they were significant predictors in the 
opposite direction than that originally hypothesized. 
The greater the number of hours of chemistry stud- 
ied by the teacher, the less likely the teacher was 
to be committed to CBA. This reverse- predictor 
operated only for the college-bound science and non- 
Science groups for the behavioral readiness mea s- 
ure. A possible rationalization for these results is 
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that teachers who were already familiar withthe con- 
cepts introduced in the CBA course had included 
these concepts in their own teaching prior to their 
institute experience. Therefore, they felt it unnec- 
essary to adopt the CBA course to supplant a course 
which already included much of the same material. 
Those teachers with weaker backgrounds in chemis- 
try, meeting certain concepts in the CBA course for 
the first time, might be more prone to adopt the new 
method in its entirety. For the behavioral read t- 
ness — college-bound science group, the second neg- 
ative predictor, professional reading and meetings 
attended emerged as significant. From the data col- 
lected in the course of this study, no logical explan- 
ation could be advanced as to why this predictor 
should have been operative in the reverse direction 
to that originally hypothesized. 


Table 3 indicates the importance of the best pre- 
dictors in explaining the variance inthe commitment 
variables, Between one-fourth to two-fifth s of the 
total variance is attributable to the pre-institute 
characteristics which are strong predictors, 


CONCLUSIONS 


A primary conclusion to emerge from this study 
was that the perceived disposition of the teacher's 
reference group toward a teaching method is a sali- 
ent predictor to the teacher’s commitment to that 
method, This result is consistent with previous re- 
searchin related areas, which lends credibility to the 


EXAMPLE 2 


SELECTED ITEMS FROM QUESTIONNAIRE II 


Attitudinal Component 
(Teachers were to respond to each item for each 
point “agreement” scale, rangingfrom strongly 


of the three groups of high school students using a five- 
agree to strongly disagree.) 


The CBA course gives the students sufficient grounding in the descriptive side of chemistry. 


The CBA course provides a stimulating experience for the majority of the students, 


The CBA course is so designed that the Students can relate 


this information in dealing with new problems. 


A laboratory, such as that presented in CBA, whi 
characteristic of a chemist is inappropriate for t 


Behavioral Readiness Component 


(A simulated situation was devised detailing a hypothetical teachin; 
groups of high school students were represented. The teachers 
questions in terms of what they would do if they were confronted 


Setting.) 


Approximately what percentage of the teaching time in the classroom situation woul 


Indicate what portion of the laboratory ex 
divisions in the CBA laboratory book. 


Would you recommend 
as a textbook? 


ch requires the students to perform the 
he students. 


previous and new learnings effectively and use 


functions 


Б Situation in which t 
were asked to respond 
with these decisions in 


he three different 
to the following 
the presented 


d you use CBA material? 


periments you would have the Students do from each of the gro 
"oup 


that the school system purchase sufficient copies of the CBA so that it could bi 
ч е used 
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TABLE 1 


RESULTS OF THE PRINCIPAL COMPONENT ANALYSES 


College- Bound 
Science Group 


College- Bound 
Non-Science Group 


Terminal High 
School Group 


Factors Factors Factors 
1 2 1 2 3 1 2 3 4 

Attitudinal Component 

Eigen- 

value 7.8 1.6 8.2 1.6 1.3 6.9 17 1.8 1.2 

Percent 

Trace 48.8 10.3 48.1 9.3 7.4 42.8 10.4 1.9 7.3 
Behavioral Readiness Component 

Eigen- 

value 5.4 5.1 4.7 

Percent 

Trace 67.7 63.8 58.7 


ТАВІЕ 2 


INDEPENDENT VARIABLES SELECTED A 


DEPENDENT VARIABLE 


S BEST PREDICTORS FOR THE SIX SCALES OF THE 


College- Bound Beta College- Bound Beta Terminal High Beta 
Science Group Weights* Non-Science Group Weights School Group Weights 
Attitudinal Measure 
tudinal Measure 
Innovativeness .40 Reference Group ‚45 Reference Group 41 
Disposition Disposition 
шы Group .30 Innovativeness .34 Innovativeness . 40 
" Isposition 
E New York  -.24 Innovativeness .21 Teach in New York -.26 
State 
Behavioral Readiness Measure 
Chemistr à 
У Hours -34 Reference Group 438 Innovativeness .38 
Innovativene. Disposition 
8S E Innovativeness „38 Reference Group 235 
Readings and Disposition 
Meetings = 
E -30 Chemistry Hours -.30 


*Onl i ith si 
Y predictors with Significant beta Weights (p - 


- 10) are reported 


TABLE 3 


VALUES FOR THE SQUARE OF THE MULTIPLE 
CORRELATIONS (R?)' OF THE BEST 
PREDICTORS FOR THE SIX SCALES OF THE 
DEPENDENT VARIABLE 


Scale R? 


Attitudinal Component 


College- Bound Science Group .31 
Attitudinal Component 

College- Bound Non-Science Group .41 
Attitudinal Component 

Terminal High School Group .39 
Behavioral Readiness Component 

College- Bound Science Group .23 
Behavioral Readiness Component 

College- Bound Non-Science Group .33 
Behavioral Readiness Component 

Terminal High School Group .24 


*R? states that proportion of the total variance 
which can be attributed to the independent variables. 


present study. Sherif (10) stated that *...numer- 
ous studies coming both from psychologists and from 
Sociologists have shown that the major source of the 
individual's weighty attitudes are the values or norms 
of the groups to which he relates himself, ' (i. e. his 
reference groups). The expectations of other groups 
(membership group, direct contact group. authority 
group, and composite groups) within the role pattern 
of the teacher were shown in this study not to be sig- 
nificant predictors for the commit ment of the teach- 
er to the CBA method, 


The second strong predictor indicated in the anal- 
yses was an innovativeness variable. Teachers rank- 
ing high on the innovativeness variable were more 
likely to evidence strong commitment to the CBA 
method, than were teachers with a lower innovative- 
ness score, This factor was conceived as an indi- 
cator of the extent to which the individual teache г 
and his immediate school environment demonstrated 
the practice of adopting new ideas within the profes- 
sional sphere. Little attempt was made in measur- 
ing this variable to separate the innovativeness of the 
teacher from the pressures toward innovativeness 


FIGURE 1 


SCHEMA REPRESENTING THE DECISION- MAKIN 
TRAINING PROGRAM 
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exerted within the school. However, since this var- 
iable was demonstrated to be an efficient predictor, 
the desirability of exploring the secondary question 
is enhanced. 


Other predictors were also shown to be significant 
for some measures of the commitment variable, but 
did not exhibit the universality of the two primary 
predictors. Predictors in this group included 
whether the teacher was employed in New York 
State, the teacher's number of earned hours of 
credit in chemistry, and the degree to which the 
teacher attended professional meetings and read 
professional journals. These last two predictors 
exhibited a reverse relationship to that which had 
been originally hypothesized. 


The results of this investigation also emphasize 
that these pre- institute characteristics of the teach- 
ers accounted for a significant portion of the vari- 
ance in the teachers' commitment to CBA. Because 
of the importance of these characteristics in the 
final decision, those sponsoring and directing sum- 
mer institutes might consider modifications in the 
institute offerings for different types of participants, 


FOOTNOTES 


1. This manuscript is based on the writer's doc - 
toral dissertation completed under the direc- 
tion of Dr. Fletcher G, Watsonand Dr, 
Maurice Belanger, “A Study of Initial Char- 
acteristics of Participants in а CBA Summer 
Institute as Predictors for FinalCommitment 
to CBA, " Graduate School of Education, Har- 
vard University, 1966. 


2. Eight items were devised to ascertain the com- 
mitment of the teacher to a particular group, 
the identification groups of the teacher, the 
aspirations of the teacher to group member- 
Ship, and the group most respected by the 
teacher. Some of these items were adapted 
from the work of Weller (11). 


3. Because of the possibility that students in the 
college- bound non-science group might be 
either continuing or terminal chemistry stu- 
dents, an additional item was included in the 
attitudinal component for this group, tt 
raising the total to 14 items, — : s 


G PROCESS OF TEAC HERS EXPOSED TO AN INSTITUTE 


Alternatives 
Range of 


Outcomes: 
Results of Applic 


ot Method 


Return to 
School 


ation 
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4. As an extension of the major рогііопої the study, 


a follow-up questionnaire was sent to the par- 
ticipants after they returned to their schools. 
The purpose of this questionnaire was to de- 

termine the degree to which these teachers 

utilized the CBA method in their own teaching. 
Failure to receive a significant percentage of 

returns made this analysis unfeasible, 
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THE USE OF EXPERIMENTAL DESIGNS IN THE 
DECISION-MAKING FEEDBACK PROCESS 


GORDON WELTY 
The Board of Public Education, Pittsburgh, Pennsylvania 


ABSTRACT 


This research paper is concerned with the relationship of feedback of research find- 


ings and with educational decision making. 


STUFFLEBEAM has argued that on methodologi- 
cal grounds, experimental designs are incompatible 
with feedback of research findings for educational 
decision making. A change in treatment during the 
evaluation cycle vitiates the research findings suffi- 
ciently to render rigorous controls useless (лун 
it is true that there is a logical incompatibility, 
then either feedback for program change in the so- 
cial action projects is impossible, or else experi- 
mental designs must be abandoned and a new 
‘theory of evaluation’’ developed. We show thathis 
contention is ill-founded. Perhaps on practical 
grounds, a design is incompetently implemented, or 
resources are lacking,etc. However,there is no ba- 
sis in principle for not using experimental designs, 
and having feedback during the evaluation cycle. 


Represent a project to be evaluated as 


Yo * (x) 


when x is input, Ф is treatment or process, and y, 
is a time independent output. The experimental de- 
sign ( e.g. Campbell and Stanley's Design 4) is sym- 
bolized by 


The former is the experimental, the latter the con- 
trol group, ( represents the effects of confounding 
Variables, constant across treatment levels. If 
"2 15 Significant there isa treatment effect, Given 
dissemination of research findings E (у) (as expected 
values of y) during the evaluation cycle, the rational 
manager reacts by changing to $*, thus on Stuffle- 
beam's argument, falsifying the evaluation report. 


Introduce a reaction function R describing the 
rational manager's response: 


$* = Rọ: 


By the contractive mappingtheorem, in a metric 
space (Ф, ), for any reactions 


Ré,, Rè, , if 
d(Rb,, ЕФ, )* kd (d,, Ф, ) 


where o «k <1, then R has a uni ue fixed poi 
such that Фе* = $, (2). This is the Stabia value of the 
treatment, where the manager will cease to modify 
the project in response to evaluative feedback, It is 
also the correct evaluative judgment, so that 


y =E (у) = & (x), 


Hence we see that both the classical е; 
design and treatment variance during th 
Cycle, as a function of feedback, can be 
of a general evaluation theory. 


Xperimental 
© evaluation 
constituents 
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COEFFICIENT ALPHA, TEST RELIABILITY,AND 


HETEROGENEITY OF SCORE DISTRIBUTIONS 


DONALD W. ZIMMERMAN 
Carleton University 


GRAHAM J. BURKHEIMER 
East Carolina University 


ABSTRACT 


From concepts which refer only to observed scores and which allow the parameters of score distributions 
over repeated measurements on a given person to differ from person to person, necessa ry and sufficient 
conditions under which coefficient alpha equals test reliability are derived. The result clarifies the relation of 
this quantity to the Kuder- Richardson formula 20, to the KR 21, tothe Spearman- Brownformula, and to Lord's 


item sampling model. 


SINCE THE original derivation by Kuder and 
Richardson (6) of the formula which Cronbach desig- 
nated as coefficient alpha theassumptions required 
have been more fully explicated(5, 4, 3, 2, 8). Der- 
ivations have been based on the classical test theory 
model in which true scores and error scores are 
uncorrelated, in which error scores оп parallel tests 
are uncorrelated, and in which the parameters of 
error score distributions over repeated measure- 
ments on a person are the same for all persons. 
The purpose of this Paper is to present a deriva- 
tion of coefficient alpha 
those which 
Teveals necessar ici ition 
der which this qu 


NOTATION AND DE FINITIONS 


A notation 
(4) will be 
taking on val 
measureme, 
taking On va 
The subscript 1 
refer to ae b 


similar to one introduced by Guttman 
The subscript Е, 
refer to repeated 
The subscript КЕ 
refer to persons. 
rom 1 to N, will 
Expectations, variances, 


oted by Е, 02, and Cov, 


ripts indicating the variables Over which 


these values are taken, 


We consider a set of pairsof independent repeated 
measurements, whichare not yet identified with any 
empirical testing procedure. One member of each 
pair is denoted by X ji the other by Кы Total 


test scores are Xj and. 


The quantity X; is considered to be a random vari- 


gj 
able having expectation over repeated measurements, 
as the number of measurements increases without 
limit given by 


and variance given by 


o2 у=к(х? )- [EKo]. 
g Б) g s F gj 


" ?0c 
Similar expressions define pe gj) and " es ei 


2 
The quantities EX...) , E(X), 0.) and 
SEK) A gil’, gji 
ot 


g ФТ) refer to the expectations and variances 


of part-test scores for person j over repeated meas- 
urements, 


E + O 
ee ee ae 
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Total observed variance is 


g 
8 


ох )= Е x? Je[ m 02 x > where 
jj 8 j^ el j 


i en 
A corresponding definition can be written foro (К gj" 


Similar expressions also define total observed part- 


2x 


25. 
test variances, di gii! and gj gi 


Total observed covariance of pairs of repeated 
measurements is given by 


СОУ (Хх; Х'.) = E(X. 
8) B 


(ХХ )-E(X JER ). 
gj E gj B gi £j 


gj gj 8 


The reliability of a test is defined as follows: 


COV (Xan X' 


gir X'gj) 


2 > CON 
gj скар Д gj 


The reliability of a part-test, рү, is defined in the 
same way, in terms of Xgji and X efi š 
It is assumed that the probability distributions of 
Xoj and X'gi are the same for any person j, such that 
E (X,.) = E (X',.) and c? (x Js о? (X). 
g 98 g D g 8 g Е 


It is not assumed that these probability distibutions 
are the same for different persons (12, in press). 
The same concept applies to the part-test scores, 
Xoji and X'i It follows from the above that 


о? (X4) =02 (X), o? fE (x ? = 02 [E(x], 
ш ^ gj (y |e a ijg 8 


and p [= tg) > E(t) | 1; шшк 47. А 


Е “ш is the product- moment correlation of 
g 


E(x ; and E (X' .) for these K persons. Likewise, 
g 8 gj 


5 


the same 


The quantity coy Oli > X'qj) is the sum of two 
Wantities, gj 


relationships hold for the part-test scores. 


91 
COV(X.,, X' ) = E[COV(X i xy] «eov Гаа), вш 
ei ge El - вро аш, y lg 8 e 


Since repeated measurements are independent, it 
follows that COV (X . > X',,) is zero for all к per- 
g gj 5) — 


sons. Therefore, 


COVE gje X) = cov Ex... E G5] = o? [ГЕ 2 Л 
gj 98 35 7 E С) Peg] ? Eea]? Fi 


The quantity о? gi) is also the sum of two quanti- 
ties, 85) 


2 2 2 
ох) - E[s? (x 5]. e? [e (x 5]. 
eal = sls | j |: "1 


Using these results, together with the equalities es- 
tablished above for pairs of measurements, a nd 
making substitutions in the definition of reliability, 
we arrive at the result 


2 Efo2(x 
г [fw] E] 


o*(x ) 
gj gj 8 


ш pa 


An equation for the reliability of a part-test score 
can be derived in the same way. We have 


2 2 

o3 E(x у Efo x.) 

j j a] 3E «i 
ру = = 1 - 


2 2 

сх) с2(х..) 
ч 1 

Qj 5 "mus 


Alternatively, if equation (1) is takenasa definition 
of reliability, as done by Guttman (4), the definition 
given in the present paper can bederivedasa theorem, 


The symbol & will denote the quantity 


N 

et sa o ect 

N-1 v? (x ) 
gj 8 


We wish to establish necessary and sufficient 
conditions under which o - P. using only the as- 
sumptions made in the present section. It should be 
noted that the definitions above refer only to Observed 
scores, Хы, that no assumptions concerning true 
ores and their correla- 
tions have been made, and that the parameters of 
score distributions over repeated measurements 
may differ from person to person. 


DERIVATION OF COEFFICIENT ALPHA 


Total observed variance can be expressed 
sum of variances and covariances of part-test 
as follows: 


as the 
Scores 


E > COVA. OX у 
i=l m=1 g,j 89 gim 


irm 
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where the subscripts i and m refer to pairsof part- 
tests. Variance of E(,;) can likewise be written 
a 


N 
T oy [Fe i E84) | 


[к 
E di SERIE Zu 


je E [cov a, т 
Ly Й Е gji 


М UD 


Since COV (Хш, Xpy - . cov [o Е) 
si m 


and since COV (X, ) is zero for all j, 


z ^g Ха 
cova, E | 
gj Sejm? 7 T СЕ е “am | à 


Subtracting equation (3) from equation (2) and using 
the relationships 


c? ) -E[c чк. ) E(X ) |and 

gi 9 i g "Ple 8 

о?(х m) = cx c? [ex )], we obtain 
ej 9 i М "j g & 

N 
2 2 N 2 

E[c*(x = = c"Xx.)-Z e" | EK,,,) 
jr a isl gj 8 іе) : w] 


Then, from equation (1) 


N 2 
Z 2 (x i] 
@ p-i-iijlg 9 


o?) 
£i) 
N N 
z ох) = 02 IE) 
= ic1g,j + i=l j g gji 
2 
с! 2 
gi Ха) Т] 


Since о“ |E(x,.,) 2 E у 2(g E(x 
Я: s i Е ай {| Н gji 
and о [E(X E fo? [g 2 
i dr gi ji j E KOJ E rote 


it follows that 
TE nete [em] - epe] {тгл 


EE 


} 


Substituting this result in (4) gives 


$ аә HEL} а Е 


На 


ох.) л] 
L2] Є S ea 


Since o? [ E ej] 


E Kj) 
[кай "uc s? fE EO] =) Le 
ile N 1 i |g м? 


Тһегеїоге, 

si s [s z о? Es j 
tile PIF le 8) = ^ 
ох.) мох) N 

gj gj 9 


Substituting p in (5) and solving for р leads to the 


following result 
И | Em 
. ааа в.) Ж 
СЕП = 
sa el 


which except for the right- hand term, 
peE 


repre]. 


In other words, coefficient alpha is equal to the re- 
liability of the test if and only if, in a population of 
repeated measurements, the mean over persons of 
the variances over part-tests of the mean part-test 
Scores over repeated measurements is equal to the 
variance over part-tests of the means over persons 
of the mean part-test scores over repeated measure" 
ments, 


vie Cow ҮЧЕ per 


Toc 
cu 
is the same 


We can write 


Ke 


We consider now conditions under which these 


quantities are equal. If, for alli, j, EK, n) = А; + 
5 
В;, where А; varies over i but is the same for all 


J 
jand B. varles over j but is the same for all i, 
= J 
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then 
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ОМА) 
1 


gi? value over the entire N x K 
matrix is the sum of a row effect, whichis invariant 
over columns, and column effect, which is invariant 
Over rows. A verbal expression of this condition is 
that the score of each person on each part-test is de- 
termined by two ‘‘effects’’ -- ‘‘ability’’ and “test 
difficulty’ ( or “ет difficulty’’), whichdo not inter- 
act. 


That is, each E(X, 
& 


Also, it follows from the above that for all i and m 


. )=COV(X х’ )2COV(X х ), 
COV Ot rji eim gj E gji gj gim  gim 
2 242 
ое i) =), 
gj S gj sim 


Pm = PK i Xgjm) » and 


p | E(X 
g 


) Ed 


ц), E(X 
i 
E g 


gjm 


In other words, the variances and covariances of all 
part-tests are equal, the intercorrelations among all 
parts are equal and equal to the reliability of each 
part, and the correlation between the EC) values 


is unity for all pairs of part tests. 


In the classical test theory model 
X үтү ES and 


g) 
H X 
^8 
т, = Шю 8 › Where T, and E аге true 
i H- = 9 j gi 


and error components of scores. In the present de- 
rivation the quantity E(X oj) has not been identified 


g 
with the concept of true score. If, however, EQ) 
g 


is a linear function of Ty the conditions stated by 


Novick and Lewis (8) are applicable and hold even 

when true anderror components of scores are corre- 
lated and when the distributions of scores over re- 
Peated measurements are different from person to 
Person (1, 9, 10, 11). That is, even in this case 
a necessary and sufficient condition for coefficient 


alpha to equal test reliability is that the part-tests 


© essentially т- equivalent, as defined by Novick 
Lewis, 


From a si vs " , 
ni = 
Tate of gle administration of a test an esti 


* is provided by the quantity 


= sx) 


sx) 
where s?) is the variance of scores on part a. 


over all К persons on a single administrationof a 
test and 52 X) is the variance of total scores over 


all К persons on a single adminstration of a test. 


If conditions for @ are met, the covariance be- 
tween pairs of items within the test equals the c ov- 
ariance between pairs of items on paralleltests. Fur- 
ther, if conditions for the KR 21 are met, for any 
given person the variance of scoresover items equals 
the variance of scores on any item over repeated test- 
ings--that is 


o2X = 02X__, for alli, j. 
gi gji gj 


When individual items are considered a is equi- 
valent to the Kuder-Richardson formula 20. That 
formula, then, is equal to the reliability of a test 
if and only if test items satisfy the above conditions. 


EQUIVALENCE OF COEFFICIENT ALPHA AND 
THE SPEARMAN-BROWN FORMULA 


The conditions under which the reliability of a 
composite test of N parts is equal to coefficient 
alpha are identical to the conditions under which the 


reliability of a test lengthened N times is given by 
the Spearman-Brown formula. The latter condition 
is sometimes expressed by saying that the parts ad- 
ded to the original test must all be parallel. Butun- 
der the usual definition of parallelism thisis a suffi- 
cient condition for coefficient alpha, while the exten- 
sion of the concept of parallelism made by Novick 
and Lewis is a necessary and sufficient condition, 
In the case in which N refers to items the conditions 
under which the reliability of a test of N items is 
given by the Kuder- Richardson formula 20 are identi- 
cal to the conditions under which the reliability of a 
l-item test lengthened N timesis given by the Spear- 
man-Brown formula. 


The relationship can be seen by letting Pi be the 
reliability of one part-test, or one item, before the 
test is lengthened. If the above conditions are met, 
N 
E ох.) = м aa), and we can substitute in 
E Rara gi 
i=l 6.] 83) 

о an equivalent expression in terms of part-test 


variances and covariances and write 


N Ld bn) к, 
в) p= 1- [23] = 
N-1 E Ld ieee 14(8-1)0 


Pi, 


These conditions can be summarized as follows. 
A sufficient condition for coefficient alpha to equal 
the reliability of a composite test of N parts is that 
the parts be parallel, in the sense in which this term 


94 THE JOURNAL OF EXPERIMENTAL EDUCATION 


has been defined in test theory. А necessary and 
sufficient condition for coefficient alpha to equal the 
reliability of a composite test is that the parts be es- 
sentially 7- equivalent,asdefined by Novick and Lewis, 
provided the E(X, i) values are linearly related to 
g 
true scores. Asufficient condition for the Spearman- 
Brown formula to equal the reliability of a test 
lengthened N times is that the added parts be paral- 
lel. Finally, a necessary and sufficient condition 
for the Spearman- Brown formula to equal the relia- 
bility of a test lengthened N times is that the added 
parts be essentially 7-equivalent. Still another way 
of expressing the condition for the Spearman- Brown 
formula is that in lengthening the test the expected 
observed score of each person on each of the added 
parts, E(Xgii), is the sum of two components, Ai 

g 


and Bj , where Av referring to the difficulty of the 

part, is the same for all persons and Bp referring 
to the ability of the person, is the same for all parts. 
RELATION OF THE KUDER-RICHARDSON 
FORMULA 20 TO THE KUDER-RICHARDSON 
FORMULA 21 


Assume that the conditions for w = pare met. 
Then, for a test consisting of dichotomous items 


N 
z EEG, 3] dil | 
mapa, Ч. |: di ST Pp Sa. 
N-1 7 
ox) 
Ё.) 
N E (Хар) [N-E хы] Е кошы} 
Б 1- 8g gj) -t 3 LE 4 
е ici 2 
N-1 No“ (X,5) ох) 
gi 6 в) 70 


When the right-hand term is zero. this result reduces 
to the Kuder-Richardson formula 21. Inother words, 
the condition required for the KR 20formula to equal 
the reliability of the test is that 


E 402 | EX5;)| (= o2 | E |E, Yi, 
di E Z m eo 


and the condition required for the KR 21 is that 


Ef o TE о : 
VOS) 


Or, for a test consisting of dichotomous items, the 
Condition for the KR 20 formula is that for alli, j 
EX ) * Ai + B, 


z gji j and the condition for the KR21 


formula is that for all i, | BE 


gj ) = Equation 


9 obi reliability of a test without restriction 
2 gji) values or on the nature of the parts. 


и s 
ái тышы isa is applicable, an estimate 
ib pre M bs a single administration of a test 


lO 


N 1- X(N - X) 
Nod Ns?(x) 


where X is the mean of total scores over К per- 
sons on a single administration of a test. 


DERIVATION OF COEFFICIENT ALPHA FROM 
ITEM SAMPLING MODEL 


An interpretation of the Kuder-Richardson for- 
mula 21 can be based on the model presented by 
Lord for the sampling of test items from a popula- 
tion of items. If Lord's expression for the standard 
error of measurement of a person's score, 


Xgj(N 7 Xgj) : 
Ni 
is identified with error variance and if its arithme- 


tic mean is taken over K_ persons and substituted in 


g 
pei-e 
о? 
х 


the classical equation, 


the KR 21 formula is obtained (7). The KR 21 for- 
mula, therefore, can be interpreted as an equation 
for test reliability for the case in which the observed 
Score distribution over repeated measurements is 
identified with variations in score resulting from 
item sampling. 


It should be noted that the requirement that 


Ef 02 [£,,) |l- te EX | t= 0 
Efo E a fre ite si 


is implicit in this model. From the concept of item 
sampling employed by Lord it follows that, as H in- 
creases without limit, for any j, ккан = ЕФ) 


for all i, such that о? Ё ( xa] becomes zero for 
i LE 


ilg 


alli, such that o?f E[E(x .)] 1-0. Also note 
7 12е 8) 


ted measure- 


allj. Further, E Ме becomes the same for 


that the distribution over independent repea ssarily 
ments of scores on dichotomous items 15 "he sat of 
binomial, apart from any identification of sampling 
repeated measurements with an empirical 9? 
process. 


i 1 е 
Let us assume now that the ЇЇ obige ft 
drawn from N distinct populations and > at T ede 
of EG) for person j may differ. Then, 
g 
5 
bution of observed scoresover repeated measurement 


is given by a Poisson sequence of trials, 


2.) = , Jie? EZ pi 
кш) = N е FL o| Egi 


Li 


Аш 
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Finding the arithmetic mean of this quantity over all 
K_ persons and substituting the result in equation (1), 
Wwe obtain (6). Therefore, equation (6) givestest re- 
liability for the case in which the N items of a test 
for K persons are sampled from NK distinct popula- 
tions of items. Given the same restrictions men- 
tioned in the last section the KR 20 formula and the 
KR 21 formula, respectively, follow from the item- 
sampling interpretation. However, the value of 


ох) for test scores consisting of the sum of 
g 

Scores on dichotomous items is given by the above 
expression and the value of p is given by equation (6), 
apart from any empirical identification of the set of 
repeated measurements. 


SUMMARY 


Beginning by considering distributions of observed 
Scores over independent repeated measurements, the 
following was proved: 


1. Coefficient alpha is equal to the reliability of 
a test if and only if, ina population of repeated 
measurements, the mean over persons of the vari- 
ances over part-tests of the mean part-test scores 
over repeated measurements is equalto the variance 
over part-tests of the means over persons of the 
mean part-test scores over repeated me asurements, 
This conclusion holds even if observed scores are 
regarded as the sumof components whichare linearly 
correlated and even if the distributions of scores 
over repeated measurements are different from per- 
son to person. This condition is equivalent to the 
assumption that the E (Xgji) value of each person on 

g 


each part-testisthe sumof two effects--one,**ability, en 
Which is invariant over items, and the other, “test 
difficulty, " which is invariant over регѕопѕ-- і.е. , 

at E 1) = А; + B, , for alli, j. 
that Е Gi) i j j 


2. The Kuder-Richardson formula 20 is equal to 
the reliability of a test if and only if test items sat- 
isfy the above equality. The Kuder-Richardson for- 
mula 21 is equal to the reliability of atest if and only 
if both of the above quantities are zero, 


reliability of atest lengthened N times is gi 
the Spearman- Brown formula; - BIER y 


4. Both the KR 20 and the KR 21 can be inter- 
preted as equations for test reliability for the case 
in which for each person the N items of a test are 
Sampled from populations defined by the conditions 
given above, Equation (6) gives test reliability for 
the case in which the N itemsofatestforK per. 
Sons are sampled from NK populations of items with- 
Out restrictions. 
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ABSTRACT 


The 388 teachers who responded represented 
агу school teachers 68.55 percent of the 566 teachers employed in the 
toward counseling апа Suidance in the e le- Participating Schools. Seven elementary Schools 
mentary schools, The motivation for Conducting 
such a project is based on the 
by research anq the 


with counseling and guid: 
contention, supported 
ory (4, 6, 1 5) h 


2 years were selected for Study and designated pro- 
" , at the ject Schools. О control Schools we. 
attitudes of teachers are most relevant, Significant rom the sam 
and valuable to Counselors and educationa 


hey Work together, to organize 
Ор, to main 


he sam e 
ach project School. attempt was 
à » to devel- made to match the project Schools w ith co ПЁго] 
tain, ang to improve school programs, Schools as to Size, number of teachers, Socio- 
economic level of the community and, in the Case 
| , , o 
p Study was designed to answer the following e community Contro] Schools, geographic ргохі рі 
Question; Some teachers in the Contro] Schools had contact with | 
Counselors Who were Providing Buidance Services to 
will teachers employed in elemen- Several schools on а part-time Schedule; however 
tary Schoo S that have maintained a they did not have the experience of Working w ith a 
Counseling and guidance program for counselor involved only with their School, 
at least years express more favor- 
able atti е5 toward counseling and A general description of the three 
п е соц 
Suidan, han teachers employed in in the same State) in Which the Project бебем се n 
elemen ary schools which have not located is Presented: sarg 
Provigeq this service? 
METHODOL осу sd One is located on the east Coast 
and is domi i { 
Тһе teachers in the selected Schools were asked dustrial interesa Бу z a al апа in- 
to comple e two attitude scales. The instruments ad- The teachers of thi а Snipping 
Minister, were modified versions of the Counseling invoked «« rofessi. М eee зА , recently 
ttitude Scale and the Scale of Attitudes Toward School Sponse to ve E reds oF 
Guidance (for a comprehensive analysis of these in- many schools ang crowded esdition of 
пе See 4, ests of reliabili; yielded i T 
ne пе of Orrelation ranging upward from . 86 Coun locateq 
visti ing Ee oed, The scores Оп the two Scales Coast and j inated p = meee 
"29 Coefficient of Correlation, ist i i ш 
tourist interest: It i major Popu 
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lation center of the state. School con- 
ditions in the county are generally good, 
although existing physical facilities are 


reaching their capacity for new students. 


Teacher salaries are the highest of the 
counties sampled. Six schools partici- 
pated from this county. 


County Three is located on the east 
coast and is not dominated by any one 
group. It has a prospering tourist in- 
dustry, a productive agriculture, and 
light industry. This isa moderately 
populated county. Although three pro- 
ject schools are situated in this county, 
à community control school was avail- 
able for only one of them. Teachers 
in the other two schools participated in 
the survey and their responses are in- 
cluded in analyses unrelated to the com- 
parison of project schools with their 
community control schools. 


In addition, elementary schools, located in ru ral 


TABLE 1 


RESULTS OF ANALYSIS OF VARIANCE 


Counties were sampled until the number of teach- 
ers selected in these schools were comparable tothe 
number chosen from the seven project schools. Eight 
rural schools from four different counties were in- 


volved. None of these rural schools had employeda 
full-time counselor. 


An analysis of variance, corrected for cells with 
unequal frequencies, was made to determine whether 
significant variances existed between the project and 
control schools’ respondents. A similar analysis 
was undertaken to determine whether the variance be- 


tween counties 1, 2, 3, and rural was statistically 
Significant. 


RESULTS AND DISCUSSION 


Teachers from project schools expressed sig- 
nificantly more favorable attitudes than teachers 
from community control Schools; however, this find- 
ing needs to be qualified because the attitude scores, 
when grouped according to county, exhibited signifi- 
cant differences between the counties sampled. This 
indicated that factors or conditions within a s chool 


Source of Variance 


S.S. df. F 
Project and Control Schools 
Counseling Attitude Scale 
Between Groups 1,207 3 2. 94* 
Within Groups 47,578 348 
Total 48,785 351 
Scale of Attitudes Toward 
School Guidance 
Between Groups 666 3 1. 06 
Within Groups 77, 655 348 
Total 78,321 351 
Counties 1, 2, 3, and Rural 
— cM ra 
Counseling Attitude Scale 
Between Groups 1, 664 * 
Within Groups 55, 479 845 adi 
Total 57,143 351 
Scale of Attitudes Toward 
School Guidance 
Source of Variance 
Between Groups 1,511 
Within Groups 72,16 3 2,82 
Total ү 


348 


74, 227 
*» <.05 351 


| 


> 


tis 


3 
AXELBERD 


TABLE 2 


DIFFERENCES AND t VALUES FOR MEAN SCORES 


Project vs. Controls 


Project vs. Community Controls 
Project vs. County Controls 

Project vs. Rural Schools 

Community Controls vs. County Controls 
Community Controls vs. Rural Schools 
County Controls vs. Rural Schools 


County 


County 1 ys. County 2 


County 1 vs. County 3 
. Rural Schools 
County 2 vs. County 3 


County 2 vs. Rural Schools 
County 3 vs. Rural Schools 


t 
t E 
2.84* 1. 46 
.86 23 
1.77 99 
2.31* 1.32 
65 49 
1.15 81 
3. 18** 2. 10* 
2.36* 2, '13** 
1. 14- .64 
."3 1.04 
12 L38 
1.05 2. 10 


*p<.05 
**p«.01 


tı - Derived from Counseling Attitude Scale, 


tg - Derived from Scale of Attitudes Toward School Guidance. 


system, other than the existing counseling and guid- 
ance programs, exerted influence on the attitudes ex- 
pressed by the teachers ( see tables 1 and 2 y 


The difference that existedbetween the express- 
ed attitudes of teachers from project and control 
schools, as well as the differences between counties 
1, 2, 3, and rural, were different in the degree of 
favorability toward counseling and guidance in ele - 
mentary schools. Since teachers from project schools 
expressed significantly more favorable attitudes (on 
the Counseling Attitude Scale) than teachers from 
community control schools, teaching in a schoolwith 
a full-time elementary school counselor is related to 
more favorable teacher attitudes than teaching in an 
elementary school which does not have a counselor оп 
its faculty. 


Another factor was found to be related to teach- 
ers’ expressed attitudes; county control respondents 
expressed significantly more favorable attitudes than 
community control respondents ( on the Counseling 
Attitude Scale ). The difference is accounted for by 
comparing the disproportional contribution of coun- 
ties 1 and 2 to the number of teachers in these con- 
trol groups. Highly significant differences, on both 
scales, in favor of county 2 over county 1 were found; 
since county 1 contributed twenty-eight teachers to 
the county controls and county 2, forty-four teachers, 
the influence of county 2 is greater than county 1 in 


the county control group. The more favorable mean 
scores of county control respondents reflects the pos- 


itive influence of teachers from county 2 in this con- 
trol group. 


The differences between the mean 
ject and control schools (although the о: 
difference was between the means of р 
munity controls), indicates anattitu 
teachers working in elementar 
a full-time counselor have more favorable attitudes 
toward counseling and guidance than teach ers 
employed in schools having more lim ited or no 
counseling and guidance programs. 


scores of pro- 
nly significant 
roject andcom- 
de trend in which 
y Schools maintaining 


Similarly, the analysis of variance be 
ties revealed a significant trend. 
Scores of county 2 on the Counselii 
and Scale of Attitudes Toward 
compared to the mean score. 
significant differences were 


tween coun- 
Whenthe mean 
ng Attitude Scale 
School Guidance were 
5 of counties 1 and 3,the 
in favor of county 2, 


CONCLUSIONS AND IMP LICATIONS 
i 


which have not Provided th 
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conclusion is qualified, in light of the signif i- 
cant differences obtained between the counties 
sampled. It is suggested that variance in teach- 
er attitudes toward counseling and guidance in 
elementary schools may be related tofactors 
other than the existing counseling and guidance 
program in a school. 


The differences between the mean scores of 
the counties sampled were statistically signifi- 
cant and the implication for educators respon - 
sible for evaluating educational programs in 
more than one county is that the evaluator be 
cognizant of the diversity of variables within a 
school system that could effect teacher attitudes. 


It is important to point out that approximately 
30 percent of the teachers asked did not choose 
to participate in the study. This 30 percent 
may include teachers who exhibit a common var- 
iable or variables related to negative as wellas 
positive attitudes toward counseling and guid- 
ance in elementary schools. 


FOOTNOTE 


This study is based on the author’s doctoral 
dissertation (1967) supervised by Dr. Rob- 
ert O. Stripling, Chairman and Professor, 
Department of Personnel Services, Univer- 
sity of Florida, 
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ABSTRACT 


Procedures are classified according to whether the dependent or independent variable sets consist of sin- 
gle or multiple variables, whether they are categorical or metric in nature, and whether or not they are statis- 


tically held constant. 


MULTIVARIATE procedures are increasingly 
being used by investigators in education and the be- 
havioral sciences. Nowadaysitis a rare study that 
deals with the relation between only two variables. 
Multiscore tests and multiple ratings are widely em- 
ployed. Therefore it seems useful to have available 
à concise symbolic classification to aid the investiga- 
tor in selecting a procedure appropriate for his data. 


Table 1 shows suchataxonomic scheme. Of course, 
Guttman's lambda (5:71-8), simple correlation, and 
the correlation ratio are not multivariate statistics, 
but they furnish a baseline against which the othe m 
more elaborate techniques canbe compared. Theela- 
borations consist of replacing a single variable bya 
set of variables and partialing out variables or sets 
of variables in various combinations. 


The third, fourth, and fifth procedures constitute 
an interesting group. Most textbooks deal with only 
the fifth, i.e., partialcorrelation. Let Xo represent 


acriterion variable (e. g. , collegiate grade point aver- 
age), X4 a predictor variable (e.g. ,scoreona stud y 
habits inventory), and х, а concomitant variable 


(е. g., scoreona scholastic aptitude test). The par- 
tial correlation is given by 


1017 102719 
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In this procedure the concomitant variable is par- 
tialed out of both the predictor and criterion, DuBois 
(4:208-10) has described an alternative technique that 
he calls ‘part correlation,” in which the concomi- 


tant variable is partialed out of only the predictor. 
The formula is 


Tor 702713 
T(0) (1. у= ———— 
= 2 
1 To 


Another possibility is to partial the concomitant 
variable out of only the criterion. The formula for 
this variant is 


Toi” t0212 
7T(0.2)0)* 7 


2 
02 


This statistic is here called ‘‘semipartial corre- 
lation, '' although the term is often used synonomously 
with part correlation. Notice that the three formulas 
differ only in their denominators. Semipartial cor- 
relation seems most appropriate for the example given 
above. It shows how much the students’ study habits 
are related to the component of their grade point aver- 
age that is independent of their scholastic aptitude. 
In Table 1 compare the paradigm for semipartial 
correlation with that for one-way analysis of co- 
variance. They differ only in the nature of the pre- 
dictor, which is categorical in the case of the latter 
In our example we could replace score on the study 
habits inventory with the dichotomy of public school 
vs. private school graduate. Then we would be com- 
paring the two types of school with regard to grade 
point average, with scholastic aptitude partialed out. 
Jennings (7) has presented a set of matrix formulas 
from which part and partialcorrelationcanbe derived 
when one or more variables are to be partialed out, 
A simple interchange of Sy mbols in Jennings’ for ў 
las covers semipartial Correlation, xd 


Е comparison is that between 
discri 
analysis (3:60-3, 116-21). In both cas redu a 


metric vari 
they occur in nature, "ADIÓS 


6 THE JOURNAL OF EXPERIMENTAL EDUCATION 


The reader may wonder at the absence of factor 
analysis from Table 1. In each of the paradigms in 
the table the variables are partitioned into two sets; 
one of the setsisregardedas being dependent on the 
other insome sense. Infactor analysis all of the vari- 
ables are treated as having the same status in a sin- 
gle set. Onecouldfollow Tatsuoka and Tiedeman (9), 
by allowing the set of dependent variables to be emp- 
ty and thus include factor analysis, Another inter- 
esting generalization is to transform the categorical 
variable(s) into sets of dichotomous variables and 
thus regard fixed effects analysis of variance as a 
regression problem (8). 


Study of the paradigms in Table 1 will show other 
interesting comparisons between the various multi- 


gle computer program, the MANOVA program of the 
University of Miami’s Biometric Laboratory (2), 
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TABLE 1 


PARADIGMS FOR MULTIVARIATE PROCEDURES 


C: a categorical variable (a single way of classifying 
people or objects) 


C’: a set of categorical variables 
M: a metric (continuous) variable 


M’: a set of metric variables 


ets 
С = (С): Guttman's lambda 


M, =): Simple (two-variable) correlation 


Mi = (Mo. M3): part correlation 
мү Е м, = f(M,): semipartial correlation 
M. Mp = (м, ж Mj): partial correlation 
M; = (M5): multiple correlation 


M = f£(M'5): canonical correlation 


M = f(M'5. M'3): multiple part correlation 


M'i М) = f(M’,): multiple semipartial correlation 


М'|. м» ={(M’3. М’): multiple partial correlation 


М = f(C): correlation ratio, one-way analysis of 
variance 


M}. Ms-f(C): one-way analysis of covariance 


M = f(C?): complex analysis of variance 

M’ =1(С): multivariate analysis of variance 

C =£(M’): discriminant analysis 

M}. M's =f(C): multivariate analysis of covariance 

м = tC): complex multivariate analysis of 
variance 


M. M, = (С°): complex multivariate analysis of 


covariance 


The period means ‘‘independent of" jn the 
sen: 
ofa least-squares residual, For examp) z 


le, the para- 
multivariate analysis of covariance ein. be 
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NOTE ON STATISTICAL DIFFERENCES 


HARL R. DOUGLASS 


Dean Emeritus 


University of Colorado 


UNTIL THE past two decades the reliability of 
statistical values was determined by the use of the 
numerical values assigned to properties of the nor- 
mal probability curve. More recently the reliabili- 
ty of differences, as well as other statistics, iscal- 
culated with chi-square and variance ratio. The 
method, however sound, is very frequently misinter- 
preted; for example, in many of the research stud- 
ies the conclusion is that there was no difference be- 
tween the control and the experimentalgroups atthe 
1 percent level, 2 percent level, or 5 percent level, 
etc. A fairly common interpretation is thatthere is 
no difference between the two groups. As a matter 
of fact, what is usually intended is that there were 
not sufficient cases to result inan obtained difference 
beyond the probability of being attributable to chance 
errors of sampling and measurement in one chance 
in 100, two chances in 100, or five chances in 100, 
In other words, the real conclusion in a great many 
of these studies is that the investigation did not em- 
ploy enough cases to permit a statistically reliable 
difference to be concluded. 


Furthermore, there is a tendency for writers of 
research projects to use ‘reliable difference” in 
such a way as to give the impression that the size 
of the difference is not important when, asa matter 
of fact, what it simply means according to the null 
hypothesis approach is that one cannot conclude with 
the small number of cases that there is really a sta- 
tistically reliable difference. The word significant 
in these studies is very often interpreted to mean 
Practical significance with respect to size or im- 
portance of the difference. As a matter of fact, all 
that has been shown is that, withthe limited number 
of cases Studied, the difference is not great enough 
that one may conclude in 95 such studies out of 100, 
or 98 such studies, or 99 such studies the differences 
of these sizes would be Obtained. There may be no 


real connection between practical and statistical dif- 
ference in many studies, 


Consider the following illustrations of artificial 
data: 


Score Male Female 
Pass 20 40 
Fail 20 20 


Chi-square = 2, 78 


Retain null hypothesis of no differences at 
5 percent level 


Score Male 
Pass 40 
Fail 40 
Chi-square = 5.51 


Reject null hypothesi i 
ler ypothesis of no differences at 5 percent 


Size of Chi-squar. i 
€ or F is not n á 
Sure of practical significanc eceSsarily a mea- 


eofan obtained difference 
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ATTITUDES TOWARD SERVICES OF 
STATE DEPARTMENTS OF EDUCATION” 


W.L. BASHAW, JAMES B. KENNEY, 
WILLIAM LANDRUM, and R. ROBERT RENTZ 
University of Georgia 


FOSTER WATKINS 


Southeastern Educational Laboratory 


ABSTRACT 


The structure of attitudes toward Services re 


states with a 70-item scale. Development procedi 


THIS STUDY was conducted as a part of theeval- 
uation of the Regional Curriculum Project, (RCP), the 
goal of which is 
ship role of state departments of education in the 
Southeast. The Regional Curriculum Project is de- 
Signed as a large, 
Twenty-four school 


systems, These sy: 
consultative, and t 


One of the criteria is attitude. 
8 toward sta Е 
partment services, An attitude s, е 


by the authors for the i 
Purpose of assessin, i 
Changes over the experimental peri Кый 


SCALE DEVE LOPMENT 
A large 


Items were clustered j 
a subjectively into tenclust 
reflecting attitude Stimuli of particular EIE the 


Project goals, Examples are “Readiness for 


ee 


ures are described. The scale was administered to randomly 
chosen samples of superintendents (N = 671), central office personnel (N 


(N = 3,684), and other local Personnel (N = 373), Data from each group 
ality. The two factors common to all groups consisted, respectively, 
despite the fact that the instrument was developed in a way intended to 


= 404), principals (N = 627), teachers 

were factored and studied for dimension- 
of positively and negatively worded items 
minimize response set factors. Athird 


Change,’’ ‘Value of Publications, ’’ “Relations with 
Higher Education, ” and so forth. Within each of the 
item clusters, items were reworded so that half of 
the cluster consisted of positively worded items and 
half were negatively worded items. A pool of 172 
items was selected for pilot testing. 


After considerable discussion, it was decided to 
use a standard 5-point Likert scale. The use ofa 
multiple choice format in which scale points were 
separately defined for each item wasa preferred for- 
mat; however, this format was discarded as being 
too bulky in terms of test materials. Items were рге" 
Sented as a series of simple, declarative sentences. 
The response choices were identical for every item 
and ranged from 


а nt; 
'*1--I almost always disagree with this sta ete Р 
or in almost all instances, this state 
false. ” 
to 


is statement; or 
'5--I almost always agree with this 5 


ы is statement is 
in almost all instances, this 
true. ” 


. y nly if 
Respondents were directed to omit an dh To baee 
they had insufficient information upon was used 
а response. This definition of Dn as indica- 
so that omission patterns could be studie 


ere 
tive of areas in which information exchanges W 
weak. 


e Uni- 
A pilot sample of 122 graduate studen Bi E z ple 
versity of Georgia was chosen. This pilo 
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consisted largely of in- service extensionclasses and 
contained representatives of all job categories of in- 
terest. Pilot data were analyzed by a series of fact- 
or analyses. The 172 items were split into three 

sets of 57, 57, and 58 items for initial factoring. 
Items were discarded on the basis of over complexity 
(high loadings on several factors) low communalities 
(implying low reliability as well as identifying specif- 
ics), and redundancy. Data were re-analyzed itera- 
tively until a final pool of seventy items was chosen. 


The pilot data yielded eight identifiable common 
factors consisting of five to thirteen items each. 
Each subset of items was reworded so that half of 
each subset consisted of positively worded items and 
half consisted of negatively worded items. The di- 
mensionality of the pilot data was determined largely 
by nature of the repeated analysis of the same data, 
and hence is not directly comparable to final results, 


SAMPLE 


All subjects are public school personnel from six 
southeastern states. State department records and 
tape listings were used to identify simple random 
samples of personnel in four job categories. The per- 
centages of persons in each job category included in 
the target sample were 


1. Superintendents 100 percent 
9. Central Office Personnel 12 percent 
3. Principals 10 percent 
4. Local School Personnel 3 percent 


These percentages were higher for the samples 
drawn from the twenty-four experimental systems. 


Test materials were distributed through the six 
state departments. Subjects returned the materials 
by mail and stamped envelopes were provided for this 
purpose. 


The percentage return ranged from 55 percent for 
local personnel to 100 percent for central office person- 
nel, The total percentage return was about 60 per- 
cent. Percentages were relatively consistent state- 
by-state. 


The category ‘‘local school personnel" was divid- 
ed into ‘‘classroom teachers” and ‘‘local personnel 
who are not teachers” by sorting obtained data. 


The sample sizes obtained were as follows: 


Superintendents 671 
Central Office Personnel 404 
Principals . 627 
Non-teacher, other local personnel 373 
Teachers 3, 684 

Total 5, 759 


The high percentage return from supervisory per - 
sons (near 100 percent in every state) was especial- 
ly pleasing since this group is the focal point of state- 
local system relationships. 


ANALYSIS 


Extensive analysis was made of the data for the 
RCP; however, this discussion will focus on ly on 
the factor analyses. Items were factored by job cat- 
egory with unities on the principal diagonal of the in- 
tercorrelation matrices and varimax rotations were 
used. 


The major problem was the determination of di- 
mensionality. Several alternatives have been sug- 
gested, the most common being Kaiser’s “little 
jiffy" criterion of using the number of factors cor- 
responding to the number of latent roots exceeding 
one. Another commonly used criterion is to take 
the number of factors that are psychologically defin- 
able. This non-analytic procedure requires multiple 
rotational analyses, but is defensible psychological- 
ly. 


Related to this second criterion is the use of the 
number of factors that appear consistently over sev- 
eral sets of data, a criterion which is defensible 
psychologically, but is often useless due to the ab- 
sence of replication data. This point-of-view has 
been used to some degree in this study. 


The authors chose to follow, as far as possible, 
Cattell’s point-of-view on dimensionality (1: 174- 
243). Cattell points out ( p. 207 ) that the Kaiser 
criterion is inadequate for large analyses. 


.. the Kaiser test. .. cuts off too soon 
when variables are few (n « 20 js 
and cuts off too late when they are 
many (n >50). 


This expectation led to the prediction that our analy- 
ses would yield many roots larger than one--an out- 


come that was obtained and is commonly obtained in 
item factoring. 


Cattell proposes his “scree test” 
analytical solution. His scree test co: 
ting the magnitude of the roots against the number of 
the root and examining the smaller roots for а 
Straight line trend. He states (p. 206 ): 


as à semi- 
nsists of plot- 


When successive roots extracted 
begin to fall in this regular way one 
is dealing only with common factors 
due to a large number of random 
small influences... 


He points out that these “random sma 


li " 
would be explained by classic theor influences 


He goes on to say ( p. 206 ): 


In large Samples, there are usually 


10 


TABLE 1 
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FIRST 25 ROOTS FOR ALL ANALYSES 


Group 
Bee Superintendent Principal Other Control Teachers Other Local 
1 14, 49 11.97 12. 97 16.41 12.49 
2 5.07 9.38 8. 76 10. 06 10.82 
3 3.37 2. 68 2.92 1. 63 2. 20 
4 1.88 1. 68 1.79 1.41 1.90 
5 1.55 1.57 157 1.30 1.65 
6 1.49 1.41 1.50 1.17 1.47 
7 1.39 1.35 1.42 .99 1. 46 
8 1.32 1.27 1.35 .97 1.39 
9 1. 28 1.19 1. 28 .96 1.34 
10 1.24 1.17 1.27 .94 1.23 
11 1.18 1.15 i. 27 ‚90 1.22 
12 1;4@ 1.10 1.19 .87 1.17 
13 1.10 1. 08 1.14 .86 1.10 
14 1.07 1.01 1.12 .82 1. 06 
15 1. 03 .99 1.10 .80 1. 03 
16 .98 .97 1.06 ‚19 1.01 
17 . 96 .95 1.01 ‚16 .97 
18 .94 ‚94 .97 .75 . 96 
19 . 93 .91 .95 ‚13 .93 
20 .91 .89 .91 т .89 
21 .89 . 88 .89 98 .88 
22 .86 .84 .88 70 .85 
ы #88 «B0 -86 ‚10 .84 
24 .82 ‚19 Aà pe di 
25 .T8 ‚178 „80 E. „16 


RESULTS AND DISCUSSION 


A partial table of roots for the 


five analy ote that in е lon 
lyses ap Note that ij Figure 1 thi root-of-one сг iter In 
Were plotted and examined for scree Slopes. Ineach large number of factors, wher easthe upper scree 
analysis, the lower Scree line was obvious. The sec line identifie very small number of r 
S a ver: 
E fact 
ors 


Опа scree line ај 


рреагей clearly in three of thefive 
Ss clearly in the other two. 


M 
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FIGURE 1 


TYPICAL SCREE GRAPH SHOWING 
THREE FACTORS 


Root Sire . 


1.0 io To ВРА 


+ з 


Root Number 


Table 1 shows the number of roots identified by 
the *'greater-than-one'' criterion. From 14to 17 


roots exceeded one in four analyses and six roots ex- 
ceeded one in the teacher data. 


The scree plots were judged as identifying three 
roots in the data for superintendents, other central 
office personnel, and principals. Two roots were 


judged to be important in the data for teachers and 
other local personnel. 


Factor similarity coefficients were calculated for 
factor matching. Matching factors general ly had 
similarity coefficients in the range .85 to . 95 . 


The factors were examined for identification. At 
least twenty-five items were found common toallfive 
analyses as identifying both factors I and П. Plotsof 
factor loadings showed that the two factors consisted 
of mutually exclusive item clusters. A typical plot 
appears in Figure 2. 


FIGURE 2 


TYPICAL SCATTERGRAM OF ROTATED 
LOADINGS ON FACTORS I AND П 


п 


The first two factors correlate іп the range of 0 
to -.3 foríour analyses. The correlationis about 
-. T for superintendents. The first two factors in 
all five sets of data define a definite response set 
splitting positively worded items and negatively 
worded items into mutually exclusive subsets. 


The third factor, identified clearly by six items 
common to three analyses, consisted of items deal- 
ing with the relationship of the statedepartments to 
institutions of higher education. The failure of lo- 
cal school people to yield factor III is most likely 
due to insufficient knowledge at this level to allow 
consistent responses to these items. 


Scales defined by items loading highly on each 
factor were analyzed for internal consistency relia- 
bility by calculating alpha coefficients for thirty job- 
by-state subgroups. Alpha coefficients for factors 
І and П were in the range . 84 to. 95, 25of the thirty 
coefficients exceeding . 90 for both factors. Factor 
Ш alpha coefficients were lower, being largely in the 
range .70 to.87. Test-retest reliabilities over a 
1-month interval were obtained on a sample of sev- 
enty-two and were all in the mid- seventies. 


Factors I and II clearly indicate that subjects re- 
sponded independently to positive and negative items. 
The authors have been unable to determine any arti- 
fact that would force this solution, Skewness in the 
distributions of item responses would not create the 
split, since the skewness has the effect of forcing 
pear-shaped joint distributions on all item pairs, 
thus affecting correlations in a fairly homogeneous 
way. The skewness certainly can affect the total 
structure by lowering correlations of item pairs that 
correlate negatively in the population when bothitems 
are either positively or negatively worded. However, 
this possibility does not seem to be related to the re- 


sult of independence between positive and negative 
items. 


, This result has also been obtained in other stud- 

ies. Some of the authors of this paper found suchan 
item split in two of several factors identified in an 

independent study of the attitudes of teachers to- 
ward their county system services, Ina personal 
communication to the senior author, Garry Foster 

(2) of the Nova School Project reports getting sever- 
al factors in children’s responses to items about their 
attitudes toward education. Two of Foster’s factors 
were negative and positive attitudes, 


A suggestion of this res 
Kerlinger’s (3) study of pro 
attitudes toward education. 
independent major attitudes a. 
namely, progressive attitud 
tudes. A review of the it 
factors suggests that the re 


А In general, 
ighly on only the 


ms loaded hi 
factor. 
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positively, negatively, or neutrally worded items 

without mixing the item types. It can also be avoid- 
ed by choosing alternate item formats such as a se- 

mantic-differential-like item format. However, such 

Scales might serve merely to hide what is perhapsa 

significant and substantive effect--a tendency to eval- 
vate independently the good and the bad aspectsof an 

educational stimulus. 


FOOTNOTES 


1. Supported by the USOE as part of the evaluation 
of the Regional Curriculum Project, 50 
Whitehall Street, Atlanta, Georgia, di- 
rected by Edward T. Brown. The conduct 
of the study and opinions expressed are sole- 
ly those of the authors and do not reflect of- 
ficial opinions of the RCP. 


2. Authors are listed alphabetically. Appreciation 
is expressed for the assistance of Edward 


1. 


T. Brown and Edith Miller, Regional Cur- 
riculum Project, H. T. Conner, Richardson 
Foundation, and the many persons ineach 
of the six states who assisted in the data col- 
lection. 
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THE RELATIONSHIP BETWEEN SELF CONCEPT 
AND ACADEMIC ACHIEVEMENT 


MORRIS D. CAPLIN 
Queens College 


ABSTRACT 
It was hypothesized that children, both white and Negro, attending a de facto segregated school have less 
positive self concepts than do children attending desegregated schools, and that there is a significant positive 


relationship between self concept and academic achievement. Sixty children from the intermediate grades of 
each of the elementary schools in a small city in northern New Jersey were matched on the basis of age, grade, 


Sex, race, intelligence, and socio-economic status. Anal 
Írom the self-report instrument administered and correl: 


y ses of variance were computed on the scores obtained 
‘ations between these scores and achievement scores were 


calculated. It was found that children attending the de facto segregated school had less positive self concepts. 
There was also a significant positive relationship between self conceptandacademic achievement. That is, those 


children having more positive self concepts had higher acad 


THE SUPREME COURT decision of 1954 outlaw- 
ing de jure school segregation recognized that 
Schools cannot be separate and equal because this 
condition contributes to psychological degradation 
and formation of a negative self-image in Negrochil- 
dren who are rejected and isolated in racially seg- 
regated schools. In recent years the meani ng of 
desegregation has been applied also to racialimbal- 
ance not due to legal discrimination. Many recent 
court decisions have found that a predominance of 
minority-group children constitutes de facto segre- 
gation. 


Various educational trends have evolved to meet 
the changing needs of our society. The compulsory 
education law was enacted to eliminate illiteracy and 
industrialization gave rise to vocational schools. 
The educator thus was faced with continuously ad- 
justing educational approaches to meet the challenge 
of a changing society. Social change, both national 
and international, has placed a new demand on edu- 
cation – providing for the culturally disadvantaged, 
particularly the Negro. The first and most impor- 
tant step may be desegregation of the de facto seg- 
regated schools. 


De facto segregated schools in some substantial 
measure characterize most public schools in north- 
егп cities (6). A study was conducted to determine 
whether children attending such a school have less 
Positive self concepts than children attending deseg- 
regated schools, and whether there is a significant 
positive relationship between self concept and aca- 
demic achievement (5), 


INTRODUCTION 


That personality factors may exerta powerful in- 


emic achievement. 


fluence on academic achievement has long been rec- 
ognized. Harris (8) compiled a list of over 300 
Studies concerned with this topic in 1940. Factors 
most frequently mentioned as importantfor aca- 
demic success were ambition, emotional security, 
a sense of responsibility, cooperauveness, and se- 
riousness. More recently, however, the signifi- 
cance of the self concept has been stressed. The 
importance of the self concept in relation to aca- 
demic learning has been given particular emphasis 
by Lecky (11). Substantial evidence indicates a re- 
lationship between self conce pt andacademic 
achievement. The literature shows that most of the 
Studies have been conducted at the high school or 
college level; the elementary school has seen rela- 
tively few investigations of this problem. 


It would seem that the relation between self con- 
cept and academic achievement Should be quite high 
and positive. Those with high achievement know it 
and those with low achievement know this too. There- 
fore, those who achieve well Should have the confi- 
dence which results from the practice ої succeedin 
and those who do not do so well do not have ag EEL 
self confidence and may not rank themselve 5 
highly. The question arises as to what the pre E: 
dent variable was, That is, do ve 


er proceeds, interprets, acce 
jects what he meets in school i 
System he has within him. 
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As measured by the instruments employed, over- 
achievers revealed significantly more positive self 

concepts. In his conclusion Lumpkin states that 

this outcome confirms current opinion that achieve- 
ment stems from intrinsic motivation, as wellas 
from the environmental responses to this achieve- 
ment. 


Based on the reported association between poor 
self concepts and reading disabilities, Wattenberg 
and Clifford (15) did an exploratory study inan effort 
to determine which was the antecedent phenomenon. 
One of the hypotheses states that the self concept is 
more important asa determinant of learning efficiency 
than learning experiences are in the for mation of self 
concept. They concluded that in general, the meas- 
ures of self concept and ego strength taken at the kin- 
dergarten level were predictive of reading achieve- 
ment two and one-half years later and that even as 
early as kindergarten, self concept phenomena are 


antecedent to and predictive of reading accomplish- 
ment at the least, 


Academic grade- point averages of children in 
grades 3 through 11 were found by Bruck (8 ) to be 


influenced by self concept. He concluded that a pos- 
itive and significant relationship exists between self 
concept and grade- point average on all grade levels 
ranging from 1 to the 5 percent level of significance, 


Bodwin (9) dida study on the relationship between 
self concept and certain educational disabilities of 
children in grades 3 through 6. He found that a pos- 
itive and very significantrelationship existed between 
immature self concept and reading disability, and a 
positive and very significant relationship existed be- 
tween immature self concept and arithmetic disabil- 
ity. 


In other related studies, Kurtz and Swenson (10), 


Nason (13), and Blackham (1) found that underachiev- 
ers had lower self concepts than did achievers. 


INSTRUMENT 


The self concept has been a concern of psychology 
for a number of years. Despite this, the subject re- 


inferences, of cour: 
about himself, 


cept definitions, 
from a study of th 


аѕ positive or negative on the basis of inter-judge 
agreement (7). A number of intermediate grade 
School children from various New York City schools 
were used as judges. Other items, especially those 
describing pupil abilities were included on the basis 
of teacher experience. The intermediate grade 
teachers were also from various New York City 
Schools and also judged the items for appropriate 
reading and conceptual levels. 


Sixty percent of the items concern the pupil's 
feelings about himself asa personand/or social be- 
ing. Forty percent relate to his feelings about school. 
Classification of itemsas personal/socialor school- 
related was made by a panel of intermediate-grade 
teachers from the school system in which this study 
was made. 


SAMPLE 


The sample group consisted of 180 intermediate 
grade children from the three elementary schools 
of a small city in northern New Jersey located near 
New York City. One of these schools was de facto 
segregated with a Negro enrollment of over 66 per- 
cent. The second school was newly desegregated by 
transferring 150 Negro children to this previously 
almost all-white school. The third school had been 
desegregated for many years due tothe housing pat- 
tern of that neighborhood. 


Since the long-term desegregated school had the 
smallest Negro enrollment (36 inter mediate-grade 
pupils) of the three schools, this group served as 
the base group. Thirty out'of the 36 Negro pupils 
were selected for the base group. The total group 
of 36 was not used because it was not possible to 
match six of these pupils on all of the control vari- 
ables to pupils intheother Schools. Thisbase group 
consisted of 15 Negrogirls and 15 Negro boys, 10 
in the fourth grade, 13 in the fifth grade, and 7 in 
the sixth grade. A similar groupof white pupils from 
the long-term desegregated school was matched to 
the base group on the basis of grade, sex, intelli- 
gence, and socio-economic status. 


Sixty childrenfrom the de facto segregated school, 
30 white and 30 Negro, werethen matched tothe base 
group again on the basis of grade, sex, intelligence, 
and socio-economic status. Finally, an equal num- 
ber, 30 white and 30 Negro from the newly desegrega 
ted school, was matched the same way. 


PROCEDURE 


Analyses of variance were computed on the prm 
representing self concept for the entire ку. Мө 
the girls separately, for the boys severe (Ж Б 
the white pupils, among the Negro pope me Di um 
the white and Negro pupils, and p bd = 
and boys. The entire instrument was t en subdivided 
into two categories and analyses of variance were 
computed as above for those items that centered pri- 
marily on the pupil’s feelings about himself ава per- 
sonal and/or social being, and for those items that 
related directly to his feeling about school, 


Correlations between the scores on the self concept 


instrument and the standard composite scores on the 
Iowa Test of Basic Skills were calculated, Correla- 
tions were also computed for the Personal/social 


e Seli-descriptiong of int 


grade i 
pupils, at w] Ch trait was designated 


CAPLIN 


qualities and for the school-rated items. 


FINDINGS 


It was found that the school-rated self concepts of 
children, white and Negro, attending the de facto seg- 
regated school were significantly lower than those of 
children attending the newly-desegregated or long- 
term desegregated schools. It was also found that 
children having more positive self concepts had high- 
er academic achievement. The correlations obtained 
between the composite standard scores on the Iowa 
Test of Basic Skills and the self concept scores on the 
entire instrument was . 52; for personal/social qual- 
ities . 45; andfor school-relateditems.58. Allthree 
correlations were significant at or beyond the . 001 
level of significance. These findings lend strong sup- 
port to the significant positive relationship between 
these measures. From this finding it could not be 
said that academic achievement is determined by the 
self concept. However, whilea correlation does not 
imply a cause-effect relationship, it does at least 
give educators and psychologists the right to look for 
a cause-effect between self concept and academic 
achievement. 


In considering the two relatively distinctive as- 
pects of self concept measured in this study, theques- 
tion arises: In relation to academic achievement, are 
the most significant aspects the generalized feelings 
and attitudes about the self, as reflected by the per- 
sonal/social scores, or are specific self perceptions, 
as reflected by the school-related scores, more sig- 
nificant? Apparently school-related traits are more 
intimately related toacademic achievement thanare 
the generalized personal/social feelings about the 
self, 


DISCUSSION 


When the significant differences between schools 
are compared, there appears to be less difference 
between the de facto segregatedand the long-term de- 
segregated schools than between the de facto segre- 
gated and the newly-desegregated schools. That is, 
the children attending the newly-desegregated school 
were significantly higher on school-related self con- 
cept. This interpretation is based upon the assump- 
tion that the children inthe newly-desegregated school 
did not have a significantly higher school-related self 
concept to begin with. Thisassumption seems to be 
reasonable since the Negro and white children who 
left the de facto segregated school were transferred 
solely on the basis of a shift in boundary lines. Thus, 
all Negro and some of the white pupils inthe sample 
group from the newly-desegregated school had attend- 
ed the de facto segregated school previously. 


This finding suggests that a certain process may 
occur when children are transferred to predominately 
white schools in order to achieve racial balance. 
First, the school-related self concept appears to go 
up. This might be due to the initial effect, a type of 
Hawthorne effect, due to the novelty of being in a dif- 
ferent situation and different school climate. Then, 
with the passage of time, the Hawthorne effect may 
decrease and school-related self concept goes down. 


What may occur is that the child finds himself in 
a desegregated school which may symbolize for him 
an opportunity. His school-related self concept goes 
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up. Now the child sees his inadequacies magnified 
in the heterogeneous classroom where most of the 
pupils are obviously better achievers. Hefinds that 
his achievement does not increase, and infact, may 
decrease as is often the case with low achievers. 
Once a pattern of low achievement has been estab- 
lished, it generally continues and becomes worse. 
This could cause his school-related self concept to 
drop again, perhaps even more severly. Thus, in 
this study involving a specific elementary school 
population, the apparent effect of desegregation was 
an initial upsurge in school-related self concept. 
The school must take positive action to protect and 
enhance this initial gain. Even though there isno 
evidence to show a direct casual relation one to the 
other, it seems likely thata higher self concept may 
contribute to higher achievement and higher achieve- 
ment in turn may contribute toa higher self concept. 


ADDITIONAL FINDINGS 


Two additional analyses of variance were com- 
puted with the data available. While these are not 
offered in support of the hypotheses, they are of in- 
terest and may provide a base for further research 
in the area. Firstwasanexamination of the differ- 
ences in self concept between the 90 girls and 90 boys 
in the sample group. No significant differences were 
obtained on any of these analyses. 


This finding may seem contrary to general expec- 
tations since it is wellestablished that girls of ages 
9-11 are physically and physiologically more devel- 
oped than boys. Also, more frequent contacts with 
women teachers, who predominate in the elementary 
school, and with their mothers may have enabled 
girls to develop a more satisfying self concept at 
these developmental stages. 


The literature on sex differences is vast, but rel- 
atively few studies have been made in the area of 
self concept in the elementary school. Some pre- 
vious studies have indicated that girls may perceive 
themselves as significantly more accepted and in- 
trinsically valued than boys (2). On the other hand 
Bruck, in his study of sex differences inthe relation- 3 
Ship between self concept and grade-point average 
in grades 3-9 concluded that there was no Significant 
sex difference in this relationshipwhenone co m- 
pares elementary pupils as a group (4). 


In a study by Piers andHarris, a wi 
Concept instrument was developed and 
to four third-grade classes, four six 
es, and four tenth-grade classes. 
ary grades, classes were chosen fro: i 
ferent schools, representing a Hann ig rig 
сіо-есопотіс levels in the community. Sex w ub. 
found not to be a significant variable (14), Е 


de-range self 
administered 
th-grade class- 
In the element- 


,. The second additional i 
tion of the differences in Men е ainas 


white and 90 N 
nificant differ 
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group being lower at the . 05 levelof significance. 


Wylie (17), inastudy concerning associationsbe- 
tween self-evaluation and several variables, found 
significant differences between white and Negro 
groups. However, these were not matched groups 
and Wylie goes on to state, ''The significant differ- 
ences which were obtained may plausibly be a func- 
tion of the occupational-level differences between 
the total Negro and white groups. '' 


The significant differences obtained in this study 
were probably not a function of occupational-level 
differences since the white and Negro pupils were 
matched on socio-economic level, However, when 
the effects of school andrace upon self concept were 
examined together, the significant difference was 
accounted for in large measure by the Negro chil- 
dren attending the de facto segregated school. No 
definite conclusions can be drawn because of the 
lack of clearly positive findings. 


SUMMARY 


The major focus of this study was academic achieve- 
ment and its relation to self concept. It was hypoth- 
esized that children, both white and Negro, attending 
a de facto segregated school have less positive self 
concepts than do children attending desegregated 
Schools; and that there is a significant positive re- 
lationship between self concept andacademic achieve- 
ment. Analysis of the data supported these hypoth- 
eses. 
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ABSTRACT 


This study was designed to answer these questions: Can useful computer programs be developed? If 
such programs can be developed, how much storage is needed and what is the cost per student? Two Fortran 


programs were developed that created and analyzed data. 


The two types of analyses were correlational analy- 


sis and a t-test of differences between means for independent samples. Each program generated and solved 
thirty problems. A sample problem and the associated statistical analysis were presented in tabular form for 
each program. The computed values of г (and t) for each student were reported. These techniques could be 
used for other types of problems without exceeding the storage of many computers. The cost was substantial. 


AS THE BURDEN onthe school becomes more com- 
plex, methods of instruction must be improved. 
Many writers (1,3, 4) would make the situation man- 
ageable by tailoring methods and materials to the 
individual learner. Others (2) place emphasis upon 
reducing the complexity of materials to fit a large 
group of students. A slightly different approach is 
researchonthe effect of unique materials with identi- 
callevels of complexity. The effect of such materials 
may improve the mastery of content; and the behav- 
iors exhibited during subsequent learning experiences 
or during normal professional activities may relate 
to course objectives. 


In an introductory course on statistical methods 
in education and psychology appropriate materials 
for individualizing instruction have been difficult to 
prepare, Ifa single textbook were adopted, it nor- 
mally included at most one or two problems that il- 
lustrate a particular technique. To lift examples 
from other texts was and is ethically questionable, 
and to develop sets of new but similar data time con- 
suming. The professor who generated suchdata then 
faced the problem of computation. Even with access 
to the speed of the electronic computer, the individ- 
ual who prepared the input data had a long tedious 
chore, A final consideration has been the willingness 
of students to prepare files of problems, As a re- 
sult, materials developed and used in the past must 
be assumed to be accessible for at least some of the 
students. 


The interface between the teaching of courses in 
statistics and utilization of the computer is an area 
not fully explored. There isa danger that we may 
use the computer as if it were only a fast desk cal- 
culator. The computer has been used extensively in 


data analysis. We may also use it to generate ran- 
dom numbers with a normal or rectangular distribu- 
tion. However, greater integration shouldbe possi- 
ble. Random numbers generated as output are sub- 
ject to analysis as raw data. If storage permits, the 
best use of computer time is to complete the analy- 
Sis during the same run as the data were generated. 
The speed of the computer may permit such a pro- 
gram to prepare unique (but similar) problems for 
each class member at a reasonable cost, These 
problems would be unique in the sense that the raw 
Scores are probably different but each set of scores 
could constitute a sample from the same universe 
Even if research had clearly defined the most effec- 
tive alternatives for the era before the advent of the 
electronic computer, the choice of alternatives fol- 
lowing its development would still be fluid. 


STATEMENT OF THE PROBLEM 


As the theoretical justification for theu - 
terials is irrelevant unless the materials рна 
ally be produced, the Scope of this study was restric- 
ted to defining statistical relationships, to comput: 
program development, to calculation of core stor: "d 
requirement, and to cost analyses. The s eic 
Questions for which answers were Sought SESS: = 


1. Canuseful computer 
c programs be di 
(The steps in sucha program would pile sa т 
erate unique raw data; to read SUE 


andthe population parameters 
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TABLE 1 


DATA GENERATED BY PROGRAM I FOR 
JOE JAMES 


JOE JAMES 


z 
“ 
ч 


1 45. 55. 
2 41. 68. 
3 61 99. 
4 35 69. 
5 51. 113. 
6 40. 66 
" 21. 58. 
8 50. 63. 
9 40. 51 
10 63. 63 
11 51. 21 
12 43, 78 
13 49. 71 
14 62. 67 
15 57. 78. 
16 4T. 53 
17 48. 74 
18 43. 80. 
19 54. 31 
20 54. 68. 
21 34, 51 
22 49. 48 
23 4T, 46, 
24 46, 61 
25 52. 82 
26 57. 74. 
27 97. 83 
28 37. 41 
29 38. 65 
30 42. 60. 
31 49. 91 
32 TL 96. 


and to repeat the process for each student in 
in the class, ) 


2. If such programs can be developed, what stor- 
age requirements are necessary? 
3. If such programs can bi 
e developed, what cost 
per student is required for ifie 
D EET Specified types of 
PROCEDURES 
TheIBM360 
Stat 


with rectangular distributions, but GA USS and GAUS 
created random numbers with normal distributions. 


Two main programs were developed. In one pro- 
gram (Program I) for each Xi Yi was functional- 


ly related to x but allowed to vary within specific 


limits. (These limits were fixed for a particular 
run, but could be changed for the next pass.) The 
data created by thistechnique were subjected to cor- 
relational analysis by the same program. In the sec- 
ond program (Program II) population means and 


TABLE 2 


ANALYSIS OF RAW SCORES GENERATED BY 
PROGRAM I FOR JOE JAMES 


JOE JAMES 
SUM OF X 1540, 00 
SUM OF Y 2130. 00 
SUM OF X. Y 104533, 00 
SUM OF X.X 76892. 00 
SUM OF Y. Y 152770. 00 
MEAN FOR X= 48. 13 
MEAN FOR Y- 66. 56 


(PEARSON PRODUCT-MOMENT) 
CORRELATION - 0. 367 


EQUATION FOR PREDICTING Y FROM X 


Ү'= 0.729Х + 31,471 
EQUATION FOR PREDICTING X FROM Y 
Х'= 0.184 Y + 35. 852 
STANDARD DEVIATIONS 
X Y 
9. 47 18. 83 
STANDARD ERRORS 
ү' x! 
17. 52 8.81 


standard deviations were allowed to vary within Бе 
scribed limits. The total sample was em En 
group X before population values for group. и: the 
created internally. (Witha small modification Hs 
Program, the number of cases in the sample cou 
also be computed internally but allowed to vary with- 
in defined limits.) This program created data appro- 
priate for a t-test of differences between two means 


for independent samples and completed the analysis. 
The raw data and the analysis were printed under the 
control of the instructions in the program. 


To determine the storage requirements of eac h 
Program the amount of core required for both the 


P. 


TABLE 3 


PREDICTED RAW SCORES FOR DATA IN TABLE 1 


Ah 


COUNTS 19 


main program and the subroutines was recorded from 
the Fortran compiler reports. 


In estimating the probable cost per student for 
computer time, each program was timed during ex- 


N x Y x' vas ecution. Thirty student name cards were prepared 
and each of the programs was used to prepare 
1 45. 00 55. 00 45. 99 64. 28 a unique problem for each student. Each program 
2 41. 00 68. 00 48. 39 61. 37 wastimedfromthe reading of thefirst program card 
3 61. 00 99. 00 54. 11 15. 95 to the printing of the last analysis. The total со st 
4 35. 00 69. 00 48. 57 56. 99 was determined by multiplying the total time in min- 
5 51. 00 113. 00 56. 69 68. 66 utes by one dollar per minute. Thecost per student 
6 40. 00 66. 00 48. 02 60. 64 is the result of dividing the total cost by thirty. 
T 27. 00 58. 00 46. 55 51. 16 
8 50. 00 63. 00 41. 47 61. 93 RESULTS 
9 40. 00 51. 00 45. 26 60. 64 
10 63. 00 63. 00 47, 47 77. 41 To illustrate the type of computer output gener- 
11 51. 00 27.00 40. 83 68. 66 ated by the two programs, representative examples 
12 43. 00 78. 00 50. 23 62. 83 are presented in Tables 1 - 7. Table 1 is a sample 
13 49. 00 71. 00 48. 94 67.20 problem for correlated data. A fictitious student's 
14 62. 00 67. 00 48. 21 76. 68 name illustrates the means of identification. In Ta- 
15 57.00 78. 00 50. 23 73. 03 ble 2 the summary of the correlational analysis is 
16 47. 00 53.00 45. 62 65. 74 for the data in Table 1, Table 3containsactual pre- 
17 48. 00 74. 00 49. 50 66. 47 dictions based upon the summary statistics in Table 
18 43. 00 80. 00 50. 60 62. 83 2. Table 4 contains the various Pearson product- 
19 54. 00 31. 00 41. 97 70. 85 moment correlation coefficients generatedfor thirty 
20 54. 00 68. 00 48. 39 70. 85 "students." In each of the problems in the 
21 34. 00 51. 00 45. 26 56. 26 seti. = 50. с = 10, ц = T0and Y,-X,«a, a, 
22 49. 00 48. 00 44, 70 61. 20 a е y sow w 7E 
23 47, 00 46. 00 44, 33 65. 74 
24 46. 00 61. 00 47. 10 65. 01 TABLE 5 
25 52. 00 82. 00 50. 97 69. 39 
26 57. 00 74. 00 49. 50 73. 03 RAW SCORES GENERATED BY PROGRAM II FOR 
27 51.00 83.00 51.16 13.03 JAMES JAY 
28 37.00 41. 00 43. 41 58. 45 
29 38. 00 65. 00 47. 84 59. 18 
30 42.00 60. 00 46. 91 62. 10 JAMES JAY 
31 49. 00 91. 00 52. 63 67. 20 x Y 
32 71.00 96. 00 53. 55 83. 24 
1 11.00 47. 00 
2 32. 00 55. 00 
3 30. 00 41. 00 
4 19. 00 46, 00 
5 15. 00 49. 00 
6 25. 00 51. 00 
TABLE 4 7 21.00 41. 00 
8 25. 00 47. 00 
CORRELATION COEFFICIENTS FROM 9 22. 00 47.00 
PROGRAM I FOR THIRTY STUDENTS 10 18. 00 43. 00 
11 16.00 54. 00 
12 19.00 
i z н z 13 21. 00 i КЫ 
14 27.00 47. 00 
24. 00 
1 .378 16 . 489 18 31.00 45. 00 
2 .326 17 .547 17 43. 00 49. 00 
3 . 641 18 ‚175 18 26.00 46. 00 
4 .536 19 .367 19 19. 00 50. 00 
5 ‚421 20 . 285 20 10. 00 44, 00 
6 .532 21 .547 21 32. 00 45. 00 
T .362 22 - 568 22 33. 00 43. 00 
8 .570 23 .354 23 42. 00 47.00 
9 . 616 24 . 398 24 31.00 42. 00 
10 . 356 28 E. 25 21.00 51. 00 
11 . 403 26 . 484 26 23. 00 49. 00 
12 .514 21 .242 27 21.00 44. 00 
13 .490 28 +160 28 18. 00 42. 00 
M 1511 29 . 604 29 48. 00 49, 00 
15 .384 30 -514 30 21.00 43. 00 
ы 53.00 
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was a random number with a mean of 20 and a stand- 
ard deviation of 20. That is, 9y.x - 20. In Table 
5, a representative example of data generated by 
the second program is presented. Table 6 contains 
the results of the analysis and population parameters 
for the data in Table 5. In this program H y and Hy 


was allowedto randomly vary between 20 апа 59, and 
Gy and оу between 2 апа 11. In Table 7 the indi- 


vidual t values may be examined. Given a critical 
value of t = 2.00, 26 of the 30 values would allow 
rejection of an appropriate null hypothesis. If fewer 
significant values were desired, a reduction in the 
range of means would give this result. 


TABLE 6 


ANALYSIS OF DATA GENERATED BY 
PROGRAM II FOR JAMES JAY 


JAMES JAY 


MEAN FOR X 24,07 
MEAN FOR Y 46.13 


STANDARD DEVIATION - X 9.12 
STANDARD DEVIATION - Y 3.77 
T -6. 60 
PARAMETER x Y 
MEAN 22 47 
5 10 5 


The program that generated correlated data 
required substantially more core storage than the 
Other program. The total required for storage of 
Program I was 20,904 bytes. Program II required 
13,687 bytes. These totals are exclusive of core 
storage required for the supervisor program, 


The total execution time for ProgramI was 
16 minutes, 13 seconds and for Program II was 9 
minutes, 44 seconds. The total cost for ProgramI 
Was $16. 25 and for Program П was$9. 75. The to- 
tal cost per student for a correlation problem was 
544, and the total cost per student for the t-test 
Problem was 33€. Additional refinement of pro- 
cedures can Slightly reduce thecost per student, but 
the operating cost ofa program should not be con- 
fused with the cost of development. 


CONCLUSIONS 


Two functional programs were developed. By 
moditying the statistical relationships, by changing 
the shape of distribution, and by varying statistical 
analyses, many types of problems can be given sim- 
ilar treatment, The amount of Storage requiredfor 
comprehensive programs is not beyond the capacity 
of many types of computers. The cost per student 
is Substantial; however, if at any time the computer 


TABLE 7 


COMPUTED t VALUES FROM PROGRAM II 


FOR THIRTY STUDENTS 


z 

= 

zx 
I> 


1 6.15 16 -1,94 
2 1.04 17 -1.87 
3 -6. 60 18 5.38 
4 6. 62 19 -2.49 
5 -7.17 20 5. 95 
6 - .28 21 A gl 
1 -1.25 22 5.67 
8 1.44 23 -6. 41 
9 -7. 07 24 4.35 

10 3.51 25 7. 67 

11 6. 93 26 -7.29 

12 5. 92 27 3.59 

13 -6. 61 28 5.71 

14 5. 79 29 6. 89 

15 -6.95 30 -1.36 


Stands idle, the real cost may just be the cost of 
operation. 


It appearsthatthese problems are different from 
most textbook problems. It is possible that problems 
with known parameters will helpto clarify the rationale 
of sampling from populations with unknown parame- 
ters. The population values are accessible to the 
professor, and he can control the timing of the ac- 
cess by students. 


Travers (5:78) implies that one who develops ma- 
terials should not attempt to evaluate such materials. 
To facilitate research neither of the two programs 
are being placed under copyright, and the author wel- 
comes correspondence on this subject. 
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ABSTRACT 


Data from 561 high-school seniors were analyzed using multiple linear regression to determine the independ- 
ent contribution of each variable to the criterion of rank inclass (R). Significant F ratioswere obtained for SCAT V 
and Q, teachers’ ratings of Accuracy (ACCU), Effort and Industry (E-I), and Reliability and Responsibility (R-R). 
R-R and ACCU, but not E-I, were found to be significant preditors of SCAT V and SCAT Q. Females were sig- 
nificantly higher than males on R, ACCU, E-I, and R-RandloweronSCAT Q. Thedatawere interpreted as sup- 
porting the effectiveness of teachers’ ratings in predicting academic performance in high school Females’ supe- 
rior performance on teacher ratings was interpreted as evidence that females conformed more than males to 


teachers’ expectations. 


AUSUBEL and others (1), found significant posi- 
tive correlations betweenteachers" ratings of adjust- 
ment, persistence, scholastic competitiveness, aca- 
demic aspirations, and the criterion of grade point 
averages in junior high school. The correlations 
were significant for both boys and girls with neither 
boys nor girls showing higher correlations in these 
attributes. 


According to the manual (Cooperative Test Divi- 
sion, 1955, pp. 3-5)the Schooland College Abilities 
Tests (SCAT) are designed for the purpose of help- 
ing teachers andcounselors estimate the capacity of 
individual students to perform work atthe next level 
of schooling. 


The items of the test are designed to measure 
school-learned aptitudes rather than general intelli- 
gence, The reason given for this is that the best 
single predictor of success from one year to the next 
is the performance observed in the preceding year, 
The manual also states that verbal and quantitative 
Skills appear to be the critical prerequisitesto next 
steps in learning in general education, 


Williamson and Cole (8) report that indices onthe 
Behavioral Differential, following the format of the 
semantic differential (5), predict academic success 
better than measures of hypothetical entities, such 


asintelligence. Their findings suggest that teachers’ 
ratings may have predictive validity if the scales are 
correctly formulated and are bipolar in nature. 


This report is concerned with: (a) Whether a 
school-generated teacher rating scale predicts rank 
in high-school class; (b) Whether SCAT scores pre- 
dict rank in class; (c) Whether SCAT and Teachers’ 
Ratings differ in their predictive validity; (d) Whether 
membership in sex grouping significantly affects 
SCAT scores, rank in class, or teachers’ ratings 
Findings which support the predictive validity of 
teachers’ ratings suggest that additional weighting 


be given to thist i i 
setting. Istypeor measurement ina high-school 


Subjects 


The subjects were members of the 


Hawaii. The students 
had complete records 


casians, and Hawaiians, were 
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TABLE 1 


TEACHERS' RATING SCALE 


ACCURACY 
(ACCU) 


Work is poor. Makes Work is inaccurate Workiswelldone 
frequent errors. and below standard. and reasonably accurate. highest quality. 


COOPERATION 
(COOP) 


80 100 


0 10 20 30 40 50 60 70 90 
Ls |. 1L 1 1 ү | { L1. 


Work is of 


1 = 


Disagreeable. Cannot Works withothers 


sometimes but has 
others. difficulty. 


or will not work with 
EFFORT- 


ar ia B 
E-I 


NN. | 


Usually agreeable. Always agreeable. f 
Generally willingto Willing to do extra 
| favors. 


sible. Lazy. 


INITIATIVE- 
LEADERSHIP 


ADEI ШИ ШШ ИИ ИИ Шо 


Does аз little as pos- Seldom completes 
required work. 


Usually does work Very industrious. 
thatis required. Oc- Does extra work 
casionally does extra gladly. 


LI Las 


Acts only under 
direction, 


Seldom originates 
any work. Follows activities and exe- think for himself. 


Plans many of his Marked ability to 


others, cutesthem. Still Carries out ideas. 
RELIABILITY- needs supervision. 
RESPONSIBILITY 
(R-R) 
Neglects promise Reliable on some Usually dependable. Thoroughly de- 
and obligations. occasions, Often Conscientious. pendable. 
PROMPTNESS- Unreliable. needs supervision. L 
PUNCTUALITY ; MS 
(P-P) Undependable, Al- Frequently late. Usually on time Always on time. 
most always late. but occasionally 
SELF- late, 
CONFIDENCE 
шы Timid. Hesitant. Appears to be over Wholesomely self- Shows superb self- 
Easily influenced. self-conscious, confident. assurance. 
Instrumentation 


The records for the graduating class were examined 
and SCAT scores, rank in class, and teachers’ rat- 
ings were recorded. These records are compiled 
during the first semester of the senior year by the 
school principally for college transcript requests. 
The typical student had been rated by five teachers, 
resulting in five ratings per scale with the range be- 
ing three ratings to six ratings (see Table 1). The 
rating system thus conformed to the criteria given 
by Rugg (6) who recommends the use of at least three 
judges and approaches the number set by Symonds 
(7) who recommends the use of eight judges. The 
ratings made by the teachers were not confined to the 
intervals of the scale such as 0, 10, or 20 but were 


placed at any point which the teacher considered ap- 
propriate, 


Procedure 


EN 


9.8, 8. 5 and 8. 9 as measured in centimeters. For 
each student, ten different scores were obtained: 
Verbal (SCAT V), Quantitative (SCAT Q), averages 
of the teachers’ ratings on the scales of Accuracy 
(ACCU), Cooperation (COOP), Effort and Industry 
(E-I), Initiative and Leadership (I-L), Reliability | 
and Responsibility (R-R), Promptness and Punctual- 


ity (P-P), and Self-Confidence (S-F), and the rank 
in class (R). 


RESULTS 


The subroutines (PERSUB) of Bottenberg and 
Ward (2) and an IBM 7040 computer were used to 
determine the extent to which each variable contrib- 
uted to the prediction of rank inclass. PERSUB uses 
the statistical techniques of multiple linear regression 
to determine F ratios and exact probability values to 
four place accuracy. It presents anefficient method 
for programming computation of the estimated prob- 
ability for any specific F value. The estimate is 
based on the actual distribution of scores and may be 
used with degrees of freedom ranging from four to 
one thousand (4). The F ratio is computed between 


m 
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a full model regression equation containing all pre- 
dictor variables under consideration and a restricted 
model, 


In the restricted model in this analysis the infor- 
mation about a particular variable is not included 
and the predictive efficiency of this equation is com- 
pared with the predictive efficiency of the full mod- 
el's equation, in which all the information for each 
variable is included. In the equation used in Table 
2 the unit vector, SCAT V, SCAT Q, and all Teach- 
er's Ratings are included in an equation aslinear 
combination of vectors to predict to the criterion of 
R. The predictive efficiency of this equation which 
includes all variables and is called the full model is 
given by the R2 value 0. 6794, A restricted model 
is then constructed holding constant, for example, 
all other variables except SCAT V. The hypothesis 
tested is that all members of the sample havethe 
same scores on all variables but different scoreson 
SCAT V and have the same expected rank in class. 


By comparing the R? of the full model to the R? 
of the restricted model the independent contribution 


of the variable SCAT V to R can be obtained, 


Restricted models for the other variables were con- 
structed and tested by comparingtheir R2 values to 
the full model's R2, 


TABLE 2 


THE INDIVIDUAL CONTRIBUTION OF TEACHERS’ 
RATINGS AND SCAT SCORES TO PREDIC TION OF 
RANK IN HIGH SCHOOL CLASS 


Source n? dt E 
Full 0. 6794 

SCAT V 0. 6689 1/551 18. 0474* 
SCATQ 0. 6723 1/551 12. 2035* 
ACCU 0. 6631 1/551 28. 0165* 
COOP 0. 6774 1/551 3. 4376 
E-I 0. 6767 1/551 4. 6407** 
I-L 0. 6786 1/551 1. 3750 
R-R 0. 6762 1/551 5. 5002* 
Р-Р 0. 6780 1/551 2. 4063 
Self-C 0. 6789 1/551 0. 8594 


Note, --Full model regression equation: SCAT V. 
SCAT Q + ACCU + COOP + E- + I-L + R-R + P- Py 
SELF-C + unit vector predicting to Rank. 


*p < ‚01 level of significance. 
**P < . 05 level of significance. 


TABLE 3 


THE INDIVIDUAL CONTRIBUTION OF TEACHERS' 
RATINGS OF ACCU, E-I, AND R-R IN 
PREDICTION TO SCAT V AND SCAT Q 


Source в? а P 
Full V 0. 2852 

ACCU 0. 2749 1/551 7, 9402* 
E-I 0. 2845 1/551 0. 5396 
R-R 0. 2621 1/551 17. 8075* 
FulQ 0. 3205 

ACCU 0. 2965 1/551 19. 4615* 
E-I 0. 3163 1/551 3. 4057 
R-R 0. 2978 1/551 18. 4073* 


Note. --Full model regression equations: ACCU + 
E-I+ К-К + unit vector predicting to SCAT V and 
SCAT Q respectively. 


* P < .01 level of significance. 


The first multiple linear regression equation in- 
cluded SCAT V and Q and all the teachers’ ratings 
in predicting to the criterion of rank in class. The 
result of this analysis seen in Table 2 showed that 
SCAT V and Q and teachers’ ratings of ACCU, E-I 
and К-К were significant predictorstoR (P< , 05). Ы 


А second and third analysis (see Table 3) were 
then performed in which teachers’ ratings of ACCU 
E-I, and R-R were used to predict to SCAT у апа 
then SCAT Q. In this analysis teachers’ ratings of 
ACCU and R-R made significant independent contri- 
bution to prediction of SCAT V (P <.01), whi le 
E-I did not. In the third analysis a similar result 
was obtained in which teachers’ ratings of ACCU 
and R-R predicted to SCAT Q (P €.01), and again 


E-I did : кул, : iu 
iu^ contribute Significantly in prediction to 


ompare males ang fe- 
ely thus pinpointing 
ose variables which 
n с ontribution i i 
tion of R. Differences on each valable: "s я 
се could 

pared for males andfemales, (f dons ded ea 
with females on teachers? ratings of ACCU ee 
and R-R, SCAT VandQandR. а 


Ні ignifi H 
ferences were Obtained for ACCU. T "ac ae i 
› R-R, an 


For SCAT Q the diffe 
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significance at the . 05 level on the other teachers’ 
rating scales. Table 5 presents the means for males 
and females on these measures. This shows that 
females are significantly higher in teachers’ ratings 
of ACCU, E-I, R-R, and R (a smaller score indi- 
cates a higher rank in class), except on SCAT Q, 


where males have significantly higher scores than 
females, 


Table 6, which presents the listing of correlations 
for all variables, indicates that a higher score on 
any variable, whether SCAT Q, Vorteacher ratings 
(variables 1-9), predictstoa higher rank on the cri- 
terion R (variable 10), Since rank in class is the 
only variable where scaling is reversed, the corre- 
lations are negative for R, 


DISC USSION 


The results of the analysis shown in Table 2 re- 
vealthat SCAT V and Q make significant independent 
iction of R, but teachers’ 


most important to l 


der would result: ACCU, SCAT V, SCAT Q, R-R, 
and E-I. Thus, 


The scalar points of ACCU are defined in terms 
of work performance, going from the low point on 
Scale from 0-25 - **Work is poor. Makes frequent 
errors," to a C level performance, 55-75- “Work 
is well done and reasonably accurate, ” tothe Band 
A ranges, 80-90, with the statement, **Work is of 
the highest quality. The high degree of prediction 


TABLE 4 


DIFFERENCES BETWEEN MALES AND FEMALES 
ON ACCU, E-I, R-R, SCAT О, SCAT V, AND R 


no a O 


Source R2 


df F 
ACCU 0, 1379 1/559 89. 3996* 
E-I 0. 1359 1/559 87. 8853* 
R-R 0. 1318 1/559 84. 8830* 
SCAT Q 0. 0092 1/559 5. 1990** 
SCAT y 0. 0062 1/559 3. 4905 
R 0. 0861 1/559 52, 6644* 


Note. --Fult model re, 
form based on bin; 
membership: Ма 
of interest, 


gression equation: General 
агу sectors denoting sex group 
le = Female predicting to criteria 


*P«.0i level of signiti 
** P «.05 level pee cance; 


of significance, 
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TABLE 5 


MEAN VALUES OF MALES AND FEMALES ON 
SCAT V AND Q, ACCU, E-I, R-R, AND R 
MALES: N = 290 FEMALES: N= 271 


Males Females 
SCAT V 441. 127 466. 254 
SCATQ 631. 534 611. 066 
ACCU 8.179 8. 923 
E-I 1.109 9. 198 
R-R 7. 850 9, 488 
R 350. 655 261. 693 


from teachers’ ratings on this scale probably reflects 
the teachers’ perceptions of how well students are 
doing in class. Thus, highquality work receivesthe 
accoladeofan A or a B, and work of lesser quality, 
less accurate work, receives lower grades, This 
particular high-school milieu, and probably most 
academic settings, stress accuracy of response as 
a criterion of greatest importance in Scholastic be- 
havior, Ability factors as would nor mally be expected 
are also important. Verbalskills and, next, quanti- 
tative skills as measured by the SCAT, have predic- 
tive significance in relationship to course content. 
E-I, which is a measure of industry, again a work- 
related measure, predictsto R. R-R which may re- 
flect a student's ability to operate conscientiously 
without immediate Supervision, predicts to R and 
Shows how the characteristic of dependability has a 
positive relationship with grades. The teachers' 
judgmentof a combination of work-related skills and 
measures of academic ability account for the predic- 
tive efficiency of these variables, 


pful and agreeable (Coop) 
dent and self-directing (I- | 


e light in 
have a signifie 


to success in high-school (see Table 1, ^ iationship' 


Rating Scale), 
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Low scores on these variables would reflect dis- 
agreeableness (COOP), timidity (S-F), and the ina- 
bility of the student to direct himself into any inde- 
pendent line of endeavor (I-L). These types of be- 
havior are perhapsa reflection of culturaldifferences 
in that these students are usually reticent since many 
of them are of Japanese ancestry. This findingthen 
does not disagree with Ausubel and others (1) that 
teachers' ratings of adjustment were significantly 
related to grade point averages. A particular type 
of school environment may be shown by this finding. 
The results indicate that the primary emphasis with- 
jn the classroom is placed upon work-related skills 
and little or no weight is given to personality variables 
as far as grades are concerned. 


Predicting to SCAT V from teachers’ ratings of 
ACCU, E-I, and R-R, we note in Table 3 that К-К 
has greater predictive significance than ACCU, and 
E-I does not make a significant independent contri- 
bution to prediction of SCAT V. Similarly, with 
SCAT Q, ACCU and R-R make approximately equal 
independent contributions to predictive effectiveness, 
while E-I does not quite reach predictive significance 
in this instance. 


E-I predicts to R, but not to SCAT V and only to 
a moderate degree to SCAT Q. Though E-Iis an 
important variable in high- school performance, E-I 
as a measure of industry is not significantly related 


TABLE 6 


to verbal aptitude scores. Teachers’ ratings of ac- 
curacy and reliability may be affected by verbal abil- 
ity, but this effect does not extend to E-I. E-I may 
be a separate dimension related to success in high- 
school, which is independent of general verbal apti- 
tude. It may therefore be a unique and important 
variable in prediction to academic success in a high- 
School setting. 


On close examination ( see Table 3) it may be 
seen that E-I predicts somewhat better to SCAT Q 
than to SCAT V. It may bethatthese three teachers’ 
ratings are also measuring personality characteris- 
tics whichfacilitate performance in quantitative skills. 
The Japanese in this population, who are character- 
ized by a high needfor achievement and, in thefemale, 
a significantly higher degree of clinicalanxiety, per- 
haps indicating a highdrive state, are superior tothe 
other largest group in this population, the Cosmopol- 
itan or mixed ancestry group in allteachers’ ratings, 
R and SCAT scores. The conscientious, accurate 
student is probably, then, of Japanese ancestry (3). 


Male-female differences indicate that females have 
significantly higher scores than males on the teach- 
ers’ rating scale variables which predict to the cri- 
terion of R. The list of means shown in Table 4 indi- 
cates that only in SCAT Q are males significantly 
higher than females. The mean differences on SCAT V 
do not reach the . 05 level of significance. Females 


INTERCORRELATIONS BETWEEN SCAT V AND Q, TEACHERS’ RATINGS AND R 


10 


1=SCATV 1.0000 0, 6762 0. 5197 0, 4930 


2=SCATQ 1.0000 0.5400 0.5057 
3 = ACCU 1.0000 0.8501 
4 = СООР 1. 0000 
5-E-I 

6=I-L 

7=R-R 

8 = P-P 

9 = S-F 

10 =R 


0.4948 0.5383 0.5095 0.4538 
0.5132 0.5329 0.5457 0.4638 0.4164 
0.8917 à 0.8787 0.8638 0.7698 0,7412 
0.9322 0.8904 0.9246 0.8099 


1.0000 0.9125 0.9365 0.8225 


0.4455 -0, 5758 
-0. 5842 
-0, 7736 
0.7769 -0,7202 


0.7535 -0.7630 


1.0000 0. 9039 0.7633 о. 8101 -0.7517 


1.0000 0. 8401 0.7229 -0, 7635 


1. 0000 0.6399 -0.6816 


1.0000 -0. 5987 


Note. --Negative correlations between R (10) and other scores 
rank in class. 


1. 0000 
lE o ANN 


arethe result of smaller R 
a values showing highs 
er 
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are significantly higher on R, indicating that even 

though males have higher quantitative abilities on the 
average, the females' superiority in teachers rat- 
ings of ACCU, E-I, and R-Rare thecritical variables 
which differentiate between male versus female per- 
formance in a high-schoolsetting. This comparison 

of male-female differences was performed on each 

variable in a separate analysis for those variables 

which made an independent contribution to prediction 

of R. 


The significantly better performance(R)of females 
in a high-school setting, even witha disadvantage in 
quantitative abilities, as shown in Table 3, probably 
relates to a personality variable, This may be ac- 
counted for by thefactthat ACCU, E-I, and R-R are 
work-related values involving accurate, precise, 
conscientious work. The difference between male 
andfemale performance Suggests that females behave 
in a manner which conforms to the values judged by 
teachers as being important to scholastic endeavors, 
These results Suggest that the female role ina high- 
School setting of this type isa more conforming one 
than that of her male classmate. Sex grouping, as 
wellas SCAT V and Q, and teachers’ ratings relat- 
ing to the quality and regularity of work output, is there- 
fore an important variable as predictor of rank in 
class in high school, 


Academic performance in high school is not the 
sole product of scholastic aptitude but isbest ex- 
plained by a combination of scholastic aptitude and 
а constellation of personality traits which affect stu- 
dents’ performance in a classroom situation, Pre- 
diction based solely on standardized tests may omit 
salient variables in prediction to academic success 
in high school. These findings suggest that school- 
generated rating schedules may have predictive val- 
idity, especially where these Scales are teachers’ 
ratings of work-related skills, Improved prediction 
of academic success in a high-school Setting may, it 
is suggested, beobtained throughtheuseof well con- 
Structed bipolar teacher rating scales in conjunc- 


tion with Standardized measures of scholastic aptitude 
Such as the SCAT, 


SUMMARY 


The records of 561 high-school Seniors were ex- 
aminedfor SCAT Scores, teachers’ ratings, and rank 
in class, Highly significant F ratios were obtained 
for SCAT V and Q, teachers’ ratings of ACCU, E-I 
one ae n their prediction to rank in class, 'н-в j 
an were found to be ѕіспіѓіс, i 
SCAT V and SCAT Q, but E E uM 


cantly to either, 


а= 


Females were higher on teachers' ratings which 
predicted to rank in class, They were not superior 
to males on SCAT V and were significantly below 
males on ЅСАТ ©. The results Suggest that much 
of the variance in high-school rank may be accounted 
for by personality factors relating to work require- 
ments in a high- school setting. Since females were 
higher on ACCU, E-I, and R-R, personality factors 
that relate to conformity to teacher expectations may 
account for their superiority in high-school perfor m- 
ance as measured by rank in class, 


FOOTNOTE 


This research was Supported by NSF funds admin- 
istered by the Research Council of the University 
of Hawaii, 
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ABSTRACT 


Computer based resource units were developed to improve the individualization of instruction through unit 
teaching. Instructional behaviors of teachers were examined to see if this kind of computer aid brought about a 
change in the way teachers teach. A pre-post two group experimental design was used in which the data were 
collected by a group of trained observers using an instruction observation tool. Pre-post mean scores were 
tested and found to be significant in five out of seven dimensions of instructional behavior. Variety and kinds of 
instructional materials and methods were also examined. The data indicate that teachers who use computer based 
resource units increase the number and improve the quality of many individualized instruction tasks. 


THE REVOLUTION which professional education 
has been undergoing the last few years involves 
changes in just about every facet of education. In- 
cluded are reexamination of educational objectives, 
restructuring of subject matter, new practices in 
staff utilization and organization, originality inbuild- 
ing design and function, diversification of instruction- 
al materials and methodology, and refinement of 
measuring devices. Most of the changes that are 
taking place seem to be related in some way to an in- 
creased awareness of the need to be in touch with 
the person of the individual in the classroom, 


The alarming rateof change, along withthe move 
toward the individualization of instruction, has served 
to complicate and confuse the decision making which 
must be done by elementary and secondary school 
personnel, Computer based resource units, which 
have been given the name “Resource Guides, "were 
developed to help the teacher make decisions in his 
complicated planning tasks.! ,? Basically, in this 
study the computer serves asa retrieval system de- 
signed to assist the teacher in his decision making 
about classroom objectives, subject matter content, 
instructional activities, materials, and measuring 
devices. The resource guide was designed primarily 
to help the teacher individualize instruction through 
unit teaching, and it was necessary to see if the in- 
Structional tasks in which teachers engage become 
more individualized as a result of using the computer 
materials, Therefore, this article reports on a 
study which attempted to get at the question: Does 
the use of resource guides cause a varied selection 
of teachers to more nearly individualize their instruc- 
tion by (1) encouraging pupils to engage in independ- 
ent thinking, (2) creating an accepting atmosphere 
inthe classroom, (3) making appropriate selection 
and use of instructional materials, (4) making ap- 
propriate selection and use of teaching methods, 


(5) motivating pupils through challenge without 
threat, and (6) sensing the needs of individuals in 
the classroom? 


To get at the question, a pre-post measure of in- 
structional behavior was recorded so that observable 
differences could be noted and objectified. To dis- 
cover whether or not these differences were statis- 


tically significant, the following null hypothesis was 
tested: 


There is no significant observable dif- 
ference between the regular instructional 
behavior of a varied selection of teachers 
and the instructional behavior of the same 
teachers recorded at a time when they are 
following plans that were developed from 
computer based resource guides. 


DESIGN AND METHODS 


In an attempt to investig: 
based resource guides up 
havior of a varied sample 
nineteen high school teachers and nir i 
grade teachers were each observed i qur 3 
trained observers using Jason's Instruction Ob d 
vation Record (IOR).* The first two observations 
representing the pre-dataof a pre-post design e 
implemented under essentially the same c. adie 
for all of the teachers. enditions 
For the post-observations, the 
ed to experimental and ш pu 


atethe effects of computer- 
оп the instructional þe- 
of classroom teache ra, 


(1) the Hawthorne effect: experimental teache 
were asked to use the computer baseq E 
Source guide, while the control teachers sog 


asked to use materials drawnfrom their usual 
Sources. 


(2) an increased awareness of their own objectives; 
both experimental and control teachers were 
asked to identify the objectives they wished to 
achieve, 


(3) an increased awareness of individual pu pil 
characteristics; both experimental and con- 
trol teachers were asked to complete a pupil 
characteristics formfor each pupilin his class, 


A further function of the contro 
against biased reporting of chan; 
havior. Observers were not to 
observing experimental or coni 


1 group was to guard 
ge in instructional be- 
ld whether they were 
trol group teachers. 


» eachteacher in the Study had only 
Thatis, eachobserver 
llfour observations of 
. The instructional be- 
and rated were those 

Seven scales of the 
IOR. 


Scale A-Attitude to Difference 

Scale B-Sensitivity to Physical Setting 
Scale C-Attitude to Students 

Scale D- Use of Instructional Materials 
Scale E-Reaction to Student's Needs 
Scale F-Use of Teaching Methods 
Scale G- Use of Challenge 


At the end of the second visit, the observers left 
a listing of forty to sixty objectives that might be ap- 
plied to the particular unit inquestion and instructed 
each teacher to selectnot more than ten that he would 
like to have his class achieve as a whole, and any 
four of these ten for each pupilintheclass. The ob- 
а eachteacher to complete one 
heck list for each pupil in the 


class. This check list covered the following charac- 


teristics: 


l. LQ. 

2. Social Class 
3. Reading Level 
4. Sex 


TABLE 1 
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Grade Level 

Interests 

Vital Statistics (family information) 
Handicap-Behavior 

Developmental Tasks 


eeippn 


The objectives and pupil characteristics were fed 
into the computer, which sorted through vastamounts 
of coded material about the unit being studied. The 
result was the productionof a resource guide - a res- 
ervoir of materials containing the following data 
relevant only to the particular class for whichthe 
resource guide was generated. 


Part A- 1. Subject matter outline 
2. Large group activities 
a. Introductory 
b. Developmental 
с. Culminating 
3. Small group activities 
4. Instructional materials 
5. Measuring devices 
Part B-1. Individual activities per each pupil 
per each of four objectives 
Instructional materials per each pu- 
pil per each of four objectives. 


2. 


Significance of difference between the pre and 
post samples and between the experimental andcon- 
trol mean differences were tested at the .05 level 
by using a t-test. 


THE FINDINGS 


The carefully balanced scalesof the IOR provide 
for the heuristically useful practice of quantifying 
each instructionalbehavior in digitalsymbols. The 
symbols range from one, representing extreme 
avoidance of individualized instruction tasks, to 
twenty, representing major involvement inactivities 


designed to further the individualization of instruc- 
tion. 


Table 1 reveals that all of the observed differ- 
ences for the experimentalgroup were in the hypoth- 
sized direction of greater individualization of in- 
struction and that six of the eight changes in instruc 
tional behaviors were statistically significant. 


с р Е Е G x 

Pre 
12.9 13.7 13.7 14.1 12.5 13.1 12.1 13.3 

Роѕі 
b 14,7 15.6 15.7 16.5 13.9 15.2 14.7 15.3 

nge +1.8 1 

Significant з SE 32.0 $2.4 41.4 2 0 
at .05 level y à Ы "T "T à 
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You will note that although scale B showed a 
greater mean gain score than scale A, the former 
was not statistically significant while the latter was. 
Each scale of the IOR provides for the observer to 
respond to “‘Insufficient Evidence’’or‘‘Inappropriate 
for this Session” as well as to the scaled behavior 
section. Onscale B, twenty-four of the thirty-eight 
teachers in a sample were not givena scaled rating 
for either the before behavior, the aíter behavior, 
orboth. So the failure to reach significance on 
scale B is in part a function of the number in the 
sample. 


The failure of scale E to reach significance is an- 
other matter. It might possibly be dueto the nature 
of the behavior rather than to an innocuous quality of 
the Resource Guide. Other instructional behaviors 
have a control-by-will quality in them. That is, it 
is more or less a simple matter of professional 
decision-making to engage in the other instructional 
behaviors. The teacher has a more or less direct 
control on these; but not so with sensitivity to stu- 
dents' needs, Students! needs are many times re- 
presented by subtle behaviors. The teacher quality 
looked for here is a skill in usinga silent language— 
à skill in FEELING communication from others. A 
pupil who squints his eyes, another who stares out 
the window, or one who talks to a classmate during 
à lecture—eachone is telling the teacher something. 
More than this, a pupil who asks a question of the 
teacher may not be verbalizing his real question at 
all. His real question might go something like this: 
“Ат I doing all right, teacher?" or “Teacher, do 
you like me?" The effective teacher must be capable 
of exercising empathy—of being sensitive to a silent 
language—a language that is conveyed on the vehicle 
of feeling and em ploy s symbolizations of emotions, 
gestures, bodily movements and facial expressions, 
Even time and space are important indicators ofcom- 
munication. Silence can indicate thought, fear, ex- 
amination of a new idea, hostility, assent, agreement 
or disagreement and many more qualities of commu- 
nication. Teachers must be capable of sensing this 
“feeling’’ communication, 


This is not the kind of ability one cancall upon if 
he has never been sensitized to it. It is a skill that 
one might develop over a period of time, but does 
not have the control-by-will quality identified in the 
other instructional behaviors, 


The pre-post change for the control group failed 
to reach significance on any of the scales except 
Scale B, Here again, the sample for this scale was 
inadequate. 


It would be of no consequence if the gain scores of 
the experimental group registered significance and 
the gain scores of the control group failed to reach 
significance unless the difference between the gain 
scores were found to be significant. Non- significant 
difference between gain scores could be found if both 
groups registered plus charges but the control group 
just missed reaching significance and the experi- 
mental group just made it. Under these conditions 
the post-scores could be so close together that even 
though one was significant andthe other was not, the 
difference between the two groups would not be a 


real difference. To be sure that this did not occur, 
the differences between the control and experimental 
gain scores were computed and found to be signifi- 
cant at the . 05 level on all scales but Scale B. 


The difference between the effect of the resource 
guide upon elementary and high schoolteachers was 
also investigated and,indicationsare that high school 
teachers are affected more. Thedata indicates that 
the reason for this seems to be that elementary 
teachers begin with more of an individualized in- 
Struction orientation and behavior. 


A quantitative analysisof methods and materials 
used and of the number of pupilquestions, comments 
and interactions supports the data reported. The 
experimental group post sessions used more mate- 
rials, a greater variety of teaching methods and elic- 
ited more pupil interaction. 


CONCLUSIONS 


The following conclusions have been drawn within 
the limitations of this study: 


1. There is a difference between the regular in- 
structional behavior of a varied selection of teachers 
and the instructional behavior of the same teachers 
recorded at atime when they are following plans that 
were developed from computer based resource 
guides. 


2. Resource guides significantly affect some 
dimensions of instructional behavior but fail to pro- 
duce significant change in other dimensions of teach- 
ing. The significant changes in instructional behavior 
of teachers who use resource guides are changes that 
increase the number and improve the quality of the 
following individualized instruction tasks: 


A. encouraging pupils to engage in independent 
thinking 


B. creating an accepting atmosphere inthe class- 
room 


С: making appropriate selection and use of in- 
Structional materials 


D. making appropriate selection and use of teach- 
ing methods 


E. motivating pupils through challen; i 
ДЕН Бе without 

F. employing a wider variety of inst: i 
materials d наш 

G. using a greater number of individual 
group methods of teaching 
groups methods 


and small 
and fewer large 


B. encouragin, 


& more pupil i; Я 
action pup: nvolvement and inter- 
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SUMMARY is another object of accusation in the fight against 


losing sight of the individual. Yet, the computer- 
based resource guide seems to provide assistance in 


The results of the present study indicate that the maximizing regard for the individual in the classroom. 


development of the resource guide may contribute 
significantly to curriculum planning and to the 
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ion is knowledge 
ecessary for effi- 
cient learning. In Part 1, four authors discuss retention 


ioi 
quence, In Part 
e authors provide scholarly insights into the 
structures of such disciplines as mathematies, science, 


English, and reading, They state or imply that the 
structure of the discipline itself, as formulate by one 
RESEARCH AND DEVELOPMENT or more scholars, has provided much of the basis for 
recent curriculum devlopment, 
TOWARD THE IMPROVEMENT Instructional materials dul ndia can be considered 
OF EDUCATION as the interface between the learner and the subject 


matter being learned. In Part ШІ, the discussion 

Ра А focuses on instructional television, on some aspects 
Oued by, Herbert J. Klausmeier and George T. of the status, defects and needs of instructional ro. 
O'Hearn. 1968 search, and on computer based instruction. Ti hing 
176 pages $5.75 softcover methods, or the interactions which occur between a 
teacher and students, have been the subject of many 

research studies in the last decades, In Part IV, two 
recent paradigms for research on teaching are pre- 
sented. One approach discusses what is commonly 
called micro-teaching and the other is interaction 
analysis, 
educational problems, Finally, in Part V, three research administrators 
point out in the Preface, the a plica- — discuss the current application of research and de- 

lion of research and development strategies [i the velopment strategies for the improvement of educa- 
i а pioneering — tional Practices. The present scene in educational 
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System. (Ме Major components of an instructiond СС And in the final chapter ан ай аге de 
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PREDICTING GRADE ONE READING 
PERFORMANCE: INTELLIGENCE VS. 
READING READINESS TESTS' 


KENNETH D. HOPKINS 
University of Colorado 


E. GEORGE SITKEI 
Los Angeles County Schools 


ABSTRACT 


Since both intelligence and reading readiness tests are widely used to predict reading success in grade 

one, both types of tests were administered to all entering first-grade pupils in two schools. Performance on 
these measures was correlated with end-of-year teacher marks and scores on a standardized reading test. Al- 
though the predictive validity coefficients for the readiness test were larger than those for the intelligence test 
onbothcriteria, the differences were not statistically reliable. When IQ was added to reading readiness scores 
the multiple correlation rose from . 61 to. 67. Addin meaningtully du 


g father occupation, Sex, and age failedto 
crease the accuracy in prediction; the multiple R's increased only to . 68. 


THE COMPARATIVE validities for predicting 
grade-one reading success of tests of intelligence 
[. j and reading readiness is an issue that has received 
/ surprisingly little attention, especially in view of the 
fact that the functions of both types of tests are high- 
ly similar for beginning first-grade pupils. Several 
studies (e. g., Dobson and Hopkins (1)) have been 
reported in which one type of test was used, although 
noncomparable samples make an evaluation of the 
comparative efficacy uncertain. Mattick (4) admin- 
istered both types of tests but used a criterion of 
success obtained after only two months of grade one, 
In addition, different schools employed different tests, 
creating a special problem, since some schools were 
ungraded. This factor would have large effects on 
obtained correlations since variability within such 
classes would be considerably reduced, Fry (2) ob- 
tained inconsistent results in predicting various 


reading criteria, althoughthe pooling of information 
from different experimental methods makes q gen- 
eralization of his results equivocal, 


PROCEDURE 


All entering grade-one pupils in two e 
schools in a lower- middle class community were 
administered both the Lee-Clark Reading Re adi- 
hess Test (LCRR) (1962 revision) and the California 
Test of Mental Maturity (CTMM) (1957 S-form) di d 
ing the first three weeks of School, the latter lowe 
always being administered last. Performances 
these tests were correlated with two independent а 
teria, Subsequently gathered near the end of Us 
Schoolyear: (à) scores on the Lee-Clark Readi е 
Test: Primer (LCRT) (Form A), and (b) end-of- ds 
teacher marks, usinga 4-category scale, In addition 


leme ntary 
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to the two predictors and two criteria, other pupil data 

were collected in order toevaluatethedegree of im- 
provement that could result from the use of multiple 
predictors. The teachers reported their final marks 
prior to being made aware of the pupil’s scores on 

the standardized reading test. Over-aged pupils 

were removedfromthe sample to avoid the confound- 
ing effects of non-promotion, etc. (Age in months: 

X = 76.8, s= 4, 1). Sincea Type I error inthis sit- 
uation is quite innocuous, « was set leniently at. 10 
in order to reduce the probability of a Type IIerror, 
the more serious in this type of study. 


RESULTS 
Table Igives the predictive valiadities of the LCRR, 


language, non-language, and total CTMM IQ's for 
the 157 pupilson whom complete data were available, 


TABLE 1 


Although the predictive validity coefficients for 
the reading readiness test numerically exceeded those 
for the intelligence tests on both criteria, the differ- 
ences proved to be insufficiently great to allow re- 
jection of the null hypothesis for either criterion. In 
predicting teacher marks, correlation coefficients 
of . 571 апа, 513 and corresponding . 95 confidence 
intervals of . 455 - . 668 and . 386 - .619 were ob- 
served for the reading readiness and intelligence 
tests respectively. The difference between the non- 
independent correlation coefficients was not signifi- 
cant (t -.90). For predicting performance on the 
standardized reading test, corresponding non-signif- 
icantly different coefficients of . 612 апа. 541 (t = 
1, 16) were observed, the larger again being associ- 
ated withthe readiness test. The. 95 confidence inter- 
vals were, 503 - . 701 and . 446 - ‚642 respectively. 


A multiple regression using both IQ and LCRR 


VALIDITIES OF VARIOUS TESTS FOR PREDICTING GRADE ONE READING PERFORMANCE 


Predictors Criteria 
(1) (2) (3) (4) (5)b (6) (7) 
Predictors 
(1) Lang. IQ .476 . 853a ‚364 .342 . 448 .497 
(2) N-Lang. IQ . 8574 .442 .422 .431 .430 
(3) Total IQ . 466 ‚445 ‚513 .541 
(4) LCRR (rs) . 934 . 571 . 612€ 
(5) LCRR (gp) .595 . 590 
Criteria 
(6) T- marks ‚148 
(7) LCRT (rs) 
х 101.8 103.4 102, 8 49.0 1.00 2.45 ВЕР 
s 16.8 17.0 14.7 10. 3 .55 .94 8.1 


a 
Spurious correlation since total IQ is an average of the lan 


Both raw Scores and 


F ай 
ings irrespective of grade placement units are 


re 
Which of these two St 


г! 


е 
Since age is, in effect, held Constant in 


Independent or е effe 
fourth decimal place. = 


IQ determinations, 


guage and non-language IQ. 


Ported to illustrate th i ility of find- 
taleris used. € the essential interchangeability 


T4". age Was computed to make this value also 


The i i : 
Partial correlation did not differ from the value given in the table until the 


* 
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as predictors yielded multiple correlations . 05 - . 06 
(corrected for bias) greater than the LCRR alone. 
Adding father occupation, sex, andage to the regres- 
sion failed to add meaningfully to the accuracy in pre- 
diction;the multiple R's only increased approximate- 
ly an additional . 01 in value. 


In view of the fact that the reading readinesstest 
did at least as well in predicting first-grade reading 
performance, it was considered to be preferable to 
the intelligence test when other relevant factors are 
considered: (a) it requires considerably less test- 
ingtime, (b) it is more easily and meaningfully in- 
terpreted, (c)theeffects of improper interpretation 
are much less serious to the pupil, and (d) it is 
less expensive. 


The primary conclusion of the study, that the 
readinesstest did at least as well in predicting first- 
grade reading performance, is consistent with those 
reported by Mattick (4) forthe Metropolitan Reading 
Test and the Lorge- Thorndike Intelligence Test; it 
is also in agreement with Hahn's (3) study in which 
the Murphy-Durrell Reading Readiness Analysis and 
the Metropolitan Readiness Test were compared with 
the Pitner-Cunningham General Ability Test: Verbal 
Series. 


e 
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FOOTNOTES 


The cooperation of Miss Mary F. Sanders and 
Mr. Robert E. Oaks, principals in the Pomona 
Unified School District, in conducting this study 
was greatly appreciated. 


REFERENCES 


Dobson, J.C., and Hopkins, K.D., ''The Relia- 
bility and Predictive Validity of the Lee-Clark 


Reading Readiness Test," Journal of Develop- 
mental Reading, 6:278-81, 1963, 


Fry, E., ‘‘Comparisons of Three Methods of 
Reading Instruction,” U. S. Department of Health, 
Education, and Welfare, Office of Education, 
Bureau of Research. Project No. 3050, 1966. 


Hahn, H.T., ‘‘Three Approaches to Beginning 
Reading Instruction--ITA, Language Arts and 


Basic Readers," The Reading Teacher, 19:590- 
94, 1966. 


Mattick, W.E.,  ''Predicting Success in the 


First Grade," Elementary School Journal, 63: 
273-76, 1963. 


THE JOURNAL OF EXPERIMENTAL EDUCATION 
(Volume 37, Number 3, Spring 1969) 
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WITH CONCEPTS AND SUBJECTS 
FROM EDUCATION ' | 
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ABSTRACT 
This research sought answers to three questions: Are factors discernible as evaluation, potency, and 
activity evident? Do scales which have been associated with one of these factors continue to register high and 
relatively pure loadings on that factor? Do adjectives traditionally labeled *'positive'' maintain this posture 
with educational concepts? Semantic Differential Scales were administered twice to elementary education ma- 
jors. The resulting matrixes were factor analyzed and rotated orthogonally. Factor I was evaluation but in- 
cluded two scales often associated with other factors. Factor II seemed to be intellectual rigor, ratherthan 
potency, Factor III seemed to be an activity dimension. In about one-half of the cases ‘feminine’ defines 
the positive end of the masculine-feminine scale. Extrapolating an SD scale's factorial content or its polarity 
from various contexts to elementary education seems unwarranted. Factoranalysis should be the first step in 
each SD data analysis. 
SEMAN " А ruc^ 
five scales exhibit cor e (SD) к adjec- The problem was to determine the factorial dy 
ST they do not exhibit com cies ende ture of SD scales that have become acco evaluation: 
Tanman OPCepts — Shaw (10) Osgood robs ability as paradigms of the three factors іегтес' prospec” p 
tman (3 (8), Husek and Wittrack ($),0 a aa potency, andactivity when they are used x concepts. 
Thus, whit A Tanka and Osgood (11) Efe tive elementary teachers to rate educate ns were 
Scale intera Subject-scale interaction ig low se . In particular answers to the following 44 
three Classic sp rns tobe high. None the take, бе ш 
and activity керер? labeled evaluation, potency, 
| Percent op more or Qe ar and usually account for juation potency, 
| ance, 1. " eva - 
| вр 56000, Suci, апа Tannenb е ч а e ee E 
ane from eee Scale-defining 18), cautioned ape iated with one 
at Я се! ctives oc 
SURE еа polarity fo concept and that the Eel 2. Do scales which have become sn continue to reg 
e aci е varied, Mn may not be tenable of the three factors listed 2Р0 ve Sings on their | 
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actors for associated with the positive end of the СА pate 
tain this posture when the scale is us¢ ш 
educational Concepts? 
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METHOD 


Seventeen concepts were selected that are directly 
related to classroom activities in the elementary 
schools. Nine of them related to the major curricu- 
lar areas: they were language arts, mathematics, 
science, social studies, teaching children, teaching 
children language arts, teaching children mathemat- 
ics, teachingchildren science, andteaching children 
socialstudies. The remaining eight concepts con- 
sisted of the defining sentences of Flanders’ (1) cat- 
egories of teacher behavior: Theteachercriticizes 
or deprecates pupil behavior with intent tochange it. 
The teacher gives directions or orders. The teach- 
er expresses or lectures about her own ideas. The 
teacher asksquestions to orient pupils to school work. 
The teacher asks questions to stimulate pupil partic- 
ipation in decision making. The teacher accepts, 
clarifies, and supports the ideas and feelings of pu- 
pils. Theteacher praisesor encourages pupils. The 
teacher justifies his own position or authority, Each 
concept was rated on 14 scales which were selected 
by reviewing the literature for SD scales which con- 
sistently exhibited high and relatively pure factor 
loadings across a variety of concepts judged by differ- 
ent kinds of subjects. Evaluative scales selected 
were good- bad, nice-awful, positive-negative, 
heavenly-hellish, optimistic- pessimistic, and happy- 
Sad. Potency scales selected were strong- weak, 
heavy-light, hard- soft, and masculine - feminine. 
Activity scales included fast-slow, active-passive, 
hot-cold, and difficult-easy. 


The S’s were seventy-one seniors in elementary 
education. The SD's were administered inan educa- 
tional foundations course. Each S responded to the 
questionnaire both before and after an 8-week student 
teaching period. (This study is a part of research 
which was designed to measure attitude changes that 
develop during student teaching. Reporting factor 
structures for both pre- and post- student teaching 
administrations in this report is done only to 
increase the dataavailable for analysis). Four ran- 
dom orders of concept presentation were used. 
The scale presentation order was selected random- 
ly, and the order of ‘‘positive’’ and ‘‘negative’’ adjec- 
tives within scales was randomized. Each ques- 
tionnaire included directions to S as suggested by 
Osgood, Suci, and Tannenbaum (8). The S’s were 
given ample in-class time to complete the question- 
naire, and every S completed every item. 


Thirty-four 14 x 14 matrices of product-moment 
correlations were produced. Each of these was fac- 
tored using principal componentsanalysis. Unities 
were used to estimate communality, and eachanaly- 
sis was followed by an orthogonal rotation to Kaiser’s 
(4, 5) Varimaxcriterion, Linear correlations were 
justified because no systematic nonmonotonic ity was 
observed among variables in the several matrices, 
While nonlinear relations undoubtedly exist among 
the variables, a linear correlation model yields a 
reasonable measure of the degree of relationship for 
a monotonic relation. 


RESULTS 


The Varimax criterion terminated the rotation 
after three factors in twelve of the thirty-four analy- 
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ses, and in no case was the rotation of more than 
five factors necessary. The proportion of total var- 
iance accounted for by thefirst three factors ranged 
from 0. 462 to 0. 663. When fourth and fifth factors 
were rotated, they appeared to be reoccurrences of 
heavy loadings on evaluative scales, or afactor 
which more frequently appeared as Factor II or III 
but was deposed in that particular analysis, or they 
seemed uninterpretable. This report will, therefore, 
be restricted to an analysis of the factor loadings for 
the first three rotated factors, 


Factor I is quite clearly the evaluative demen- 
sion, All six of the scales chosen because they ex- 
hibited high and relatively pure loadings on the eval- 
uative dimension in other studies yieldedloadings 
> 0.30 in 29 or more of the 34 factor analyses 
summarized here. In addition the scale active- passive 
qualified in 32 of the 34 cases and the scale strong- 
weak qualified in 31 of the 34 cases. 


Factor II is best defined by the scales hard-soft 
and difficult-easy. In only four cases out of 34 does 
one of these scales meet the 0, 30 criterion while the 
other scale does not. Moreover, іп 19 cases out of 
the 30 in which they are paired they ranked first and 
second, 


Factor III is defined by the following scales: 
fast- slow, masculine-feminine, and hot-cold. In 11 
cases out of 34 all three scales meet the 0. 30 crite- 
rion. In an additional 11 instances, two out of the 
three met the criterion. 


TABLE 1 


FREQUENCY OF OCCURRENCE OF FACTOR 
LOADINGS ^ 0.30 ON FACTORS FOR WHICH 
SCALE IS NOT LISTED 


Relative frequency 
with which scale 
loads > 0.30 on 
other factors 


Factor I scales 


happy- sad 


) 0. 18 
active-passive 0.16 
good-bad 0. 07 
heavenly-hellish 0. 43 
positive-negative 0. 09 
nice-awful 0.16 
optimistic- pessimistic 0.16 
Strong-weak 0.21 

Factor II scaies 
hard-soft 
difficult-easy e Е 

Factor Ш scales 
fast-slow 

= я 0. 37 
maseuline-teminine 
Ue. 0. 12 
Pe NUN DA 
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Data in Table 1 provide information relative to 
the factorial ‘‘purity’’ of the high- loading scales for 
Factors I, II, and III across the set of education- 
al concepts studied. 


If we were to reject all scales with loadings 
> 0. 30 on other factors in 25 percent or more of the 
cases, then heavenly-hellish, fast-slow, and hot- 
cold would be eliminated. 


CONCLUSIONS 


Active-passive, the traditional paradigm of the 
activity factor, is clearly an evaluative scale with 
these educational concepts and subjects. In only 
three cases out of 68 does it load above 0. 50оп Fac- 
tor II or Ш. Similarly Strong-weak, a traditional 
potency scale, becomesan evaluative scale. In only 
two cases does it load above 0. 50 on Factor II or III. 
Factor II isa hybredof the traditionalactivity and po- 
tency factors. In light of the Concepts rated and the 
factor loadings it appears that hard- soft and difficult- 
easy become synonomous, and that Factor IL is as- 
Sociated with intellectual rigor, profundity, sub- 
Stance, and opaqueness rather than physical hard- 
ness, strength, orthelike, The Shift in connotative 
meaning among Factor II scales may accountfor the 
masculine-feminine scale's low loadings onthis fac- 


tor. While masculine-feminine may link with hard- 


Soft for physical attributes, it certainly does not 
form such a link for intellectual attributes connected 
with these educational concepts. Inthese settings it 
is linked with two traditional activity scales in 

Factor III. 


While masculine-feminine is the “purest” scale 
in Factor III, it does not maintain a stable polarity 
across the concepts included. An indication of this 
may be observed in Table 2. 


TABLE 2 


COMPARISON OF THE POLARITIES OF PAIRS OF 
SCALES WHOSE LOADINGS WERE > 0.30 ON 
FACTOR Ш 


Same Opposite 
Polarity Polarity 
masculine-feminine 
with fast- slow 10 6 
masculine-feminine 
with hot-cold 8 10 
fast-slow with 
hot-cold 27 2 


These data suggest that in about one- half of the 
de feminine’ defines the positive end of the scale, 
onsidering the predominance of female teachers in 


elementary school classrooms, this outcome is not 
surprising, 


It seemsclear that some scales which have come 
to be regarded as activity or potency scales because 
of their relatively consistant performance in many 
Studies did not perform in expected ways with these 
educational concepts and subjects. Kerlinger (1964) 
quotes Osgood as suggesting that SD's should always 
include scales of known factorialcontent. This study 
emphasizes that it is dangerous to suppose that a 
scale has a “known” factorialcontent that can be as- 
Sumed at the outset. Moreover, one scale's polarity 
Switched from concept to concept for the same sub- 
jects during a single SD administration. Erroneous 
results and conclusions would be generated by scor- 
ing and analyzing responses based on assumed scale 
performance, 


The 34 factor analyses including rotations requir- 
edonly 3.4 minutesonan IBM 7094 computer. Those 
who use the SD with educational concepts should per- 
form factor analysis as a first step in data analysis. 


FOOTNOTES 


l. The work reported herein was performed pursu- 
ant to a grant from the U.S. Office of Education, 
Department of Health, Education, and Welfare, 


to 


- Loadings > 0.30 for factors I, П, and III are 
presented in the Final Report, Project No. 7- E-053, 
Grant No. OEG-1-7-070053-1908, Use of the Se- 
mantic Differential Technique to Measure Pros- 
pective Elementary School Teacher Attitude 
Toward Mathematics and Other Subjects, Feb- 
ruary, 1968, USOE, Bureau of Research, pp. 11-18. 
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Robert E. Clasen 


O N TO THE CLASSROOM deals 
with typical problems common to 
teachers of disadvantaged preschool 
children and contains unique suggestions 
for understanding and meeting the needs 
of these youngsters. The chapters are 
based on papers by well-qualified pro- 
fessors mia rofessionals from the 
preschool education field which were 
originally presented to a group of Head 
Start teachers needing help in the various 
areas covered. The editor says, "Since 
these works were extremely useful to one 
group of teachers, they should be 
useful to others." 


The book begins with a chapter which 
defines "culturally deprived" and offers a 
frame of reference for the thoughts and 
ideas presented in the remainder of 

Мы the book. Each chapter was selected 
by Dr. Clasen on one criterion: 
Does it contain information which our 
experience has shown that teachers need? 
The chapters speak for themselves: 


200 pages $7.25 Hardcover and $5.75 Softcover 


Creating a Learning Environment (numerous hints are given on 
how this learning climate can be created) (Chapter 3j 


The Teacher, The Child and Head Start (the needs of children and 
à teacher's awareness are discussed) (Chapter 3) 


Speech Language Acquisition and Language and Head Start (deal 
with language diagnosis and teaching strategies) (Chapters 4, 5) 


From a Teacher's Point of View (a humorous and heart-rending day 
to day account of organizi g, canvassing, and parent program- 
ming in Head Start, plus the happenings in a Head Start 
classroom from the first class day to the last—all taken from a 
teacher's log with her commentary and suggestions) (Chapter 6) 


A Conversation with а Head Start (A.D.C.) Mother (reveals wh; 
the mother of a Head Start child experiences) (Сыры ay “Pat 


Programming for Parents (offers surprising views on what mn 
is all about) (Chapter 8) ЕМЕ vies Suey Hg 
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ACADEMIC ATTITUDES AND PERFORMANCE 


AS A FUNCTION OF DIFFERENTIAL GRADING 
SYSTEMS: AN EVALUATION OF PRINCETON'S 


PASS-FAIL SYSTEM' 


MARVIN KARLINS?, MARTIN KAPLAN, and WILLIAM STUART 


Princeton University 


ABSTRACT 


An empirical investigation of the Princeton University pass-fail system is presented. Findings based on 
questionnaire and academic performance data indicate that the pass-fail grading alternative is not the panacea 
that some educators envisioned. It is Suggested that further study of the pass-fail system be undertaken before 
it becomes a permanent grading procedure in our Universities. a 


IN THE PAST decade many American educators 


through a process akin to Allport’s notion of Func- 
have displayed increasing hostility toward the com- tional Autonomy. The relative advantages and/or 
petitive (A-F) grading system prevalent in most disadvantages of competitive grading as а pedagogi- 
Schools. These critics base their Opposition on sev- cal technique remain to be defined within the context 
eral assumptions, two majoronesbeing: (a) students of empirical research; the point is, such research 
will learn more in their courses when the deleterious is vitally needed now -- before changes in education- 
effects of competitive grading are reduced or elimi- al practice take place on the basis of intellectual 
nated; (b) encouraging learning through incentives hunch rather than scientific merit, The major aim 
Such as grades is not in keeping with contemporary of this investigation is to provide some of that nec- 
educational philosophy which says a student should essary research. 
Study because he is interested in the subject matter, 


not a grade in the subject. These educational assump- 
tions are not without theoretical support and, indeed, ance in course work, 
are congruent with current Psychological notions 


about the intrinsic motivational correlates of 
learning. 


with examining perform- 
and attitudes toward such work, 


ng Systems that differ in their 
properties (competiti 


individuals opposed 


marking systems 
› 1 It will be emphasized that а pedagogical innovation at 
to the A-F grading system has been of sufficient in- Princeton University, allowing the student the choice 
tensity to bring about a partial de-emphasis on com- of taking a course graded in the traditional (A-F) 
Petitive grading in several major universities. “пем”? (pass-fail) manner permits one to stud tor 
Princeton, Rutgers and Dartmouth, for example,have the first time, the differential performance Pls 
just instituted ( in a specified number of courses )a dents taking concurrent] пате with by Stu. 
Pass-fail alternative tothe more rigorously competi- and “‘less competitive” gradi t ариу» 
tive A-F grading system. The interesting aspect of Srading systems, 
such a change is that there is very little evidence to Before presenting the findings of the Study i4 ; 
Justity it--it might be that some Students learn less necessary to describe the Princeton Pass-fai], d 
( and are less interested in learning ) whentheir mo- For many years Princeton undergrad aH System, 
tivation to learn is ‘intrinsic’? (interest in the course graded on a competitive highly-deli vis Ye been 
material) rather than “extrinsic” (interest in the marking system employ in ITa 5 neater numerical 
course grade). One might even argue that incentives est) to 7 (worst). Pg 196 istinctiong m 14 
M) ап be Шле as the vehicle for bringing fied the standard grading procedure асу modi- 
‘© an intrinsic appreciation of learning dent the option of taking one course eod each stu. 
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pass-íail basis (for example, a freshman could elect 
to take one of his required ten courses pass-fail).To 
avoid making the pass-fail student a ‘‘second-class 
citizen," instructors were not informed whichclass 
members were taking the course pass-íail, and fur- 
ther, were instructed to treat all students alike: pass- 
fail students were expected to do the same work and 
be graded in the same way as a student taking the 
course in the more traditional manner. Only when 
the numerical grades for all class members were 
turned into the registrar were the numerical grades 
of the pass-fail students transformed into a" P" 
(5- or better) or “F™“ (бог 7). In assuring that 
each student would be treated similarly regardless 
of his grading choice (pass-failor 1-7) the Univer- 
sity provided the groundwork for a research design 
that met the four criteria necessary for studying aca- 
demic performance as a function of different mark- 
ing systems: (1) students could choose between a 
competitive and pass-fail grading alternative when 
signing up for a course; (2) theteacher inthecourse 
did not know which students were taking the course 
pass-fail; (3) the course requirements were the 
same for each student regardless of the grading sys- 
tem he selected; (4) theteacher marked all students 
on the basis of the competitive 1-7 system. 


This study represents an examination of the pass- 
fail system as it existed during the 1966-67 academic 
year (it has undergone modification since then) at 
Princeton University and reports findings based on 
questionnaire and performance data from the under- 
graduate population. Believing that it is just as im- 
portant to discover what the students think about 
the pass-fail system as it is to examine their per- 
formance within it, the investigation wascarriedout 
in two phases: the assessment of attitudes question- 
naire phase and the academic performance phase. 
These two phases will be reported separately. 


Phase 1: The Pass-FailQuestionnaire: There 
are several ways to discover the advantages and lim- 
itations of various grading procedures. One of the 
most direct and relevant ways is to solicit the оріп- 
ions of students who have experienced them. This 
method has the obvious advantage of gaining informa- 
tion directly ‘гот the horse’s mouth." If the stu- 
dent claims he learns more ina course graded pass- 
fail than in a similar course taken for A-F credit, 
one is spared the time necessary to arrive at that 
answer indirectly (through inferencefrom other data 
sources). The major drawback of thequestionnaire 
technique is the possible dishonesty of the responding 
subject who might have a vested interest in the con- 
tinuation of the pass-fail system. This individual 
might respond as he thought the University would 
wish him to respond, rather thanas he actually felt. 
Several steps were taken to encourage honesty in 
questionnaire responding: (1) instructions empha- 
sized respondent anonymity and the fact that his an- 
swers would in no way influence the pass-fail system 
at Princeton; (2) undergraduates were in charge of 
questionnaire distribution and utilization, thus under- 
Scoring student (not University) concern with the 
data; (3) one hundredquestionnaires were filled out 
by a random sample of pass-fail students ina person- 
al interview situation conducted by a fellow-student. It 
was felt that the respondent would be more likely to 
give a candid, honest view of the pass-fail system 
in the presence of an undergraduate colleague who 
assured him the answers he gave would in no way 


affect the pass-fail system than ina mailed question- 
naire which stated the same information in writing. 3 


As a check for respondent honesty оп the ques- 
tionnaires, the results obtained on the one hundred 
personal interview questionnaires were compared 
with the findings from the other protocols (906 ques- 
tionnaires from students who had voluntarily complet- 
edand returnedthemby mail) * А secondcheck for 
honesty was accomplished by giving another group of 
one hundred pass-fail students a disguised question- 
naire designed to solicit pass-fail information while 
appearing to serve a different purpose. These question- 
naires were distributed and collected before the pass- 
failquestionnaires were delivered. Ascanbe seen by 
comparing the disguised questionnaire(see Appendix 
B) withquestion 35 of the pass-fail questionnaire 
(see Appendix A) one has a means of estimating a 
person's judgment of his pass-fail course as а func- 
tion of varying instructions ( realization vs. non- 
realization that the questions are concerned with atti- 
tude toward pass-fail courses), Judging from the: 
(1) similar response patterns between subjects who 
filled out the questionnaire throughthe mailandin a 
personal interview situation; (2) close correspond- 
ence between responses by subjects onthe disguised 
and non-disguised questionnaires;and(3)the rather 
severe respondent criticisms directed at some as- 
pects of the pass-fail system in both the personal in- 
terview and mailed questionnaire sample (comments 
that would not be made by a student intent on **pleas- 
ing the University") it seems relatively safetoclaim 
that the questionnaire results present an accurate, 
honest picture of the way students feel concerning 
the pass-fail system at Princeton. 


The pass-fail questionnaire (a fifteen minute, 
36-item test including true-false, multiple choice 
and fill-in questions) was designed to assess the re- 
spondents: (1) support-non-support for the pass- 
fail system; (2) reasons for exercising the pass- 
failoption; (3) perception of the properties of the 
pass-fail system; and (4) comparisonof severalfac- 
ets of the pass-fail vs. numerical grading system 
An examination of the students’ responses should help 
answer two questions: (a) just how effective is the 
pass-fail system as a pedagogical tool?; (b) are the 
goals of the pass-fail system (the purposes for which 
it was created) being accomplished? ш 


graduates 
fail grading 
The findings in- 
Быр Fu favorably 
nly З perce; 
spondents felt the pass-fail system shouldbeaboln a 
ought more Courses 
Sis. Further, two. 
pass-f i? l system in favor of tae pata Mie 
program (e.g., instead of five с 
pass-fail; four courses, all for ed 


Table 2 presents findin: 
between a student's own 


ses, one taken 
erical grade), 


gs that allow comparisons 


program was **to encour pass-fail 
i ` age course se 
basis of interest rather than grade seco the 
ions’--~ 
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TABLE 1 


ATTITUDES OF PRINCETON UNIVERSITY 
OF THE PASS-FAIL GRADING SYSTEM * 


THE JOURNAL OF EXPERIMENTAL EDUCATION ; 


UNDERGRADUATES CONCERNING THE CONTINUATION 


Question 


The pass-fail system should be abolished at Princeton 
More courses should be graded on a pass-fail basis 


The pass-fail system should be dropped in favor of a 
course reduction program (e. g. › instead of five 
courses, one taken pass-fail; four courses, all for 
numerical grade), 


True False No Opinion 
3 92 5 
48 31 21 
| 
22 66 12 


* 
All answers are given in percentages, and are based 
tionnaire samples (N- 1,006). 


yet, only 28 percenttooka course pass-fail for that 
reason, and even fewer (7 % ) believed the interest 
factor influenced their classmates’ decision to take 
a course pass-fail. Instead, 35 percent and 37 per- 
cent of the respondents respectively listed “ to re- 
duce the tension of, and emphasis on, course grades” 
and ‘‘to provide additional Study time for other cour- 
ses and/or extra-curricular activities” as the rea- 
sons they took their course pass-fail. These same 
reasons accounted for 93 percent of the responses 
(34 % and 59 % respectively ) when the students 
predicted their classmates’ reasons for taking 
а course pass-fail. Thus, it seems that the ma- 
jority of students exercise their pass-failoption (and 
believe their classmates do likewise) to reduce grad- 
ing tensions and increase additional study time in 
other courses, rather than on the basis of course in- 
terest (the reason,they thought,accounted for the Uni- 
versity’s implementation of the pass-fail program), 


Two of the assumptions made by some critics of 
numerical (competitive) grading systems is that stu- 
dents will learn more, and their motivation to learn 


TABLE 2 


WHY TAKE A COURSE PASS-FAIL? A COMPARISON O 


on the combined random and mailed ques- 


will be higher, in pass-fail compared to numerically 
graded courses. Information relevant to these be- 
liefs is presented in Table 3. The findings do not 
Support the critics; to the contrary, the findings 
(from multiple choice questions) indicate that stu- 
dents believe they: (a) learn the same (45%) or 
less (41%) ina pass-fail than comparable numeri- 
cally graded course; (b) have a decrease in motiva- 
tion to learn (24%) or no change in motivation to 
learn (63%) when comparing pass-fail with numeri- 
cally graded courses; and (c) work closer to their 
capacity (7276) ina numericaly graded course 
rather than one marked pass-fail. 


When the student is asked to make Specific per- 
centage comparisons between certain aspects of 
pass-fail and numerically graded Courses, the find- 
ings obtained with multiple choice questions are rep- 
licated. Utilizing a percentage figure based on the 
averaged responses of all Subjects it was found that 
a student believes he works to his capacity 67 per- 
centof the time ina numerically graded course, where- 
asina pass-fail course he does so only 44 percent of 


F A STUDENT'S OWN REASONS WITH THOSE 


HE ATTRIBUTES TO HIS UNIVERSITY AND FELLOW CLASSMATES * 


Ss perception of 
the University's 


Ss perception of 
the reason he 


Ss perception of 
the other students’ 


Reasons reason for im- took a course reason for taking 4 
plementing pass- опа pass-fail @ counseypass~ 
fail basis fail 
(1) To reduce the tension of, and 
emphasis on, course grades, 10 35 Se 
(2) To Provide additional study time 
for other courses and/or extra- 
curricular activities, 3 37 
(3) To encourage course selection | 
on the basis of interest rather | 
than grade Considerations, 8" 28 7 


*AlL answers are 
Samples (N=1,006) 


aa 


given in perc, 
à Percentages, and are based on the combined random and mailed questionnaire 
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TABLE 3 


ATTITUDES OF PRINCETON UNIVERSITY UNDERGRADUATES CONCERNING HOW MUCH THEY LEARNED 
AND THE LEVEL OF THEIR MOTIVATION TO LEARN IN PASS- FAIL. COMPARED WITH NUMERICALLY 
GRADED COURSES * 


Multiple Choice Questions 


% Responding Question 


As far as the amount of material learned is concerned, I 
have learned: 


(a) much more 

(b) more 

(c) the same 

(d) less 

(e) much less, in my pass-fail than a comparable numeri- 
cally graded course. 


i 


In my pass-fail course (in comparison to numerically graded 
courses) I have had: 


13 


(a) an increase in motivation to learn 
24 


(b) a decrease in motivation to learn 
(c) no change in motivation to learn 


Ifeel that I work closer to my capacity: 


(a) in a numerically graded course rather than one graded 
pass-fail 

(b) in a pass-fail course rather than one graded numerically 

(c) about equally in both types of courses 


Hi 


Fill-in Questions ** 
.FiH-inWuestions ** 


In a numerically graded course I work to my capacity 


67 % of the time; in my pass-fail course Ido so 
44 "b ofthe time. 


In a numerically graded course I learn 74% ог the material 
presented; in my pass-fail course Ilearn 61% , 


Ina numerically graded course I am interested 70 95 of the 


time; in my pass-fail course I am interested 57 % of thetime, 


True - False Questions 


Question 


True False No Opinion 

Iwas able to learn more in my pass-fail course because it was 
not graded on a 1-7 basis. 12 69 19 
If I was taking my pass-fail course for a normal 1-7 grade,I 
would be more interested in the course in general. 21 "4 5 
If I was taking my pass-fail course for a normal 1-7 grade,I 
would gain more knowledge. 38 48 14 
If I was taking my pass-fail course for a normal 1-7 grade, I 
would gain more intellectual stimulation, 14 73 

13 


* All answers are given in percentages, and are based on the combi i 
ples (N=1,006). ined random and mailed questionnaire sam- 


** The figure in the blanks represents the average numerical response for all respondents 
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the time. Similar results were obtained when the stu- 
dent compared numerical and pass-fail graded courses 
in terms of material learned (74 % to 61 %) and inter- 
est generated (70% to 57%). 


An interesting reversal of opinions in favor of 
the pass-fail grading alternative occurred on three 
true-false items where students indicated that if they 
were taking their pass-fail course for anormal 1-7 
grade they would not: (a) be more interested in the 
course in general (74%); (b) gain more knowledge 
(48% ); (c) gain more intellectual stimulation (13%). 
Onafourth true-false item, however, 69 percent of the 
respondents answered false to the item: ‘“‘Iwas able 
to learn more in my pass-failcourse because it was 
not gradedonal-7basis.'"' A possible explanation 
for the discrepant results with the true-false items 


is that they required student estimations based on a 
hypothetical situation, whereas the multiple choice and 
fill-in items dealt with actual student behavior in the 
classroom. Considering the majority of student re- 
sponses on the majority of questions asked, it seems 
reasonable to assert that students believethey learn 
more, work closer to their capacity, and are more 
motivated to learn in a numerically graded course 
than one marked pass-fail. 


Based on findings reported in Table 3 one might 
predict that a student's level of participation in a 
course would be function of whether he was taking 
the course pass-fail or for numerical grade. The 
findings reported in Table 4 lend credence to that 
prediction and indicate that students participate more 
fully in a course when faced with competitive ( 1-7) 


TABLE 4 


ATTITUDES OF PRINCETON UNIVERSITY UNDERGRADUATES CONCERNING COURSE PARTICIPATION IN 
PASS-FAIL AND NUMERICALLY GRADED CLASSES * 


True-False Questions 


Question True False No Opinion 
If I knew my course was being graded 1-7 rather than pass-fail, 8 
I would work harder. 79 18 

Taking a pass-fail course has allowed me more time to study n 
for my other courses. 74 22 

As far as I can determine, most of the students have used pass- 

fail to underemphasize one course while devoting more time to 

their numerically (1-7) graded courses. 87 8 5 
If I had more time I'd work harder in my pass-fail course. 53 42 5 
If I was taking my pass-fail course for a normal 1-7 grade,I 

would do more of the assigned readings. 61 35 4 
If I was taking my pass-fail course for a normal 1-7 grade, I 

would attend more of the lectures and/or precepts. 42 54 4 
If I was taking my pass-fail course for a normal 1-7 grade, I 

would do better work on papers, labs, etc. 50 31 19 
If I was taking my pass-fail course for a normal 1-7 grade, 

I would study harder for course quizzes and exams. 74 22 4 


Fill-in Questions T 
Ina numerically graded (1-7) course I do 80 % of the assigned readings; in my pass-fail course I do 61%. 


In а numerically graded course I attend 85 7b of the lectures and/or precepts; in my pass-fail course I 
attend 74 % , 


* All answers are given in 


percent; i А А А 
samples (N= 1,006). ages, and are based on the combined random and mailed questionnaire 


жж i i 
The figure in the blanks represents the average numerical response for all respondents. 
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TABLE 5 


ATTITUDES OF PRINCETON UNIVERSITY UNDERGRADUATES CONCERNING NUMERICAL ( 1-7) 
AND PASS-FAIL GRADING * 


True-False Questions 


Question True False No Opinion 


I have a good idea of how hard I have to work to get a 
passing grade in my pass-fail course 72 23 5 


I have tried to do as well grade-wise in my pass-fail 
course as in my numerically graded courses 24 73 3 


The competition for grades in courses stimulates me to 
work harder 56 39 5 


If I was taking my pass-fail course for a normal 1-7 

grade, I would be under more tension due to worry over 

grades 85 12 3 
Fill-in Questions ** 


Considering the amount of work done in your pass-fail course, what numericalgrade would you expect? 3.22. 


What kind of grade would you aim for if you were taking the pass-fail course for credit? 2.17 


* All answers are given in percentages, and are based on the combined random and mailed questionnaire 
samples (N - 1, 006). 


** The figure in the blanks represents the average numerical response for all respondents, 


rather than pass-fail grading. A majority of ques- In line with these considerations, 74 percent of the 
tionnaire respondents agreed that if they were taking students agreed that taking a pass-failcourse had al- 
their pass-fail course for a normal 1-7 grade they lowed them more time to studyfor their other 
would: (a) work harder; (b) do more of the assigned Courses; and an even higher percentage ( 87 %) be- 
readings; (c) do better work on papers, labs, etc. ; lieved that most of their classmates had utilized th 
and (d) study harder for coursequizzesand exams. pass-fail system to underemphasize one cour х9 
They further stated that in a numerically graded while devoting more time to their numerically (1-7) 
course they did 80 percent of the assigned readings graded courses. In Summarizing the findings report- 
and attended 85 percent of the lectures; whereas in ed in Table 4, one can say that а majority of students 
their pass-fail course they did only 61 percent of the believed they more actively participated in numeri- 
assigned readings and attended only 74 percent of the cally graded (in comparison to pass-fail) course : 
lectures. The finding concerning attendance at lec- choosing to utilize their limited time allotment ji 
tures is somewhat contradictory to the claim of 54 courses where competitive marks Were assign ae 
percent of ne pemp who answered false to the Even if one assumes that le 
uestion: ' was taking my pass-fail course for > , at а student ici 
donat 1-7 grade, I would attend more of the lec- ld actively in competitively graded сорыс ан 
tures and/or ргесерїз.”? action rent still be expressed ov hot 


Why should a student decrease his participation 


i i i t е to worr 

in а pass-fail course? One reason might be an insuf- competiti Y Over grades i; 
ficient amount of time to distribute among his classes, € aL merked course, this tension might 
In the study, 53 percent of the students claimed they tion for grades ü percent stated that the competi- 
would work harder in their pass-fail course if they S stimulated the А 


had more time, When time is at such a premium - As might be expected 


, i Stu 
pass-fail courses must Compete with numerically mune Wired harder for grades > a 
graded courses for a student S attention--and under one-fourth of the speHtively marked, Less tha 
these conditions itis most likely thatastudent will well gradewise in fis pondents attempted to do n 
withdraw time from his pass-failclass and invest it in numerically grad d eir Pass-fail Course as in Ed 
the “‘dividend-paying” numerically marked course, ering the amount or woe ects, When asked ze 
one inyour pass- 7 
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what numerical grade would you expect?" the aver- 
age grade answered was 3. 22. Toa second question: 
**what kind of grade would you aim for if you were 
taking the pass-fail course for credit?’ the average 
grade reply was a healthier 2. 17 (a large difference 
on the Princeton grading system). Findings from 
Table 5 indicate that students believe working for 
competitively assigned 1-7 grades can be tension in- 
ducing, but that such grading can also stimulate the 
student to work harder in the course. 


Table 6 presents some general observations by 
Princeton undergraduates on various aspects of 
the pass-fail system. It is interesting to note that 


À one-thirdof the respondents foundthe sub- 
а )  pass-failcourse more difficult than 
aterial in most of the courses they took for а 1-7 
ade. This would seem to indicate that the difficulty 


the subject matter should not be a factor if it is 
mnd that students receive lower grades in their pass- 
ail subjects than in courses taken for a numerical 
(1-7) grade. 


Based on undergraduate questionnaire data,one can 


make the following summary statements (based on 
majority student opinion) concerning the Princeton 


TABLE 6 


pass-fail system: (1) Princeton students аге over- 
whelmingly in favor of retaining the pass-fail grad- 
ing alternative; (2) the majority of students exercise 
their pass-fail option (and believe their classmates 
do likewise) to reduce gradingtensions and increase 
additional study time in other courses, rather than 
on the basis of interest in the course subject matter; 
(3) students generally believe they learn more, work 
closer to their capacity, and are more motivated to 
learn in a numerically graded course than one marked 
pass-fail; (4) students believe they participate more 
fully in a course when faced with competitive (1-7) 
rather than pass-fail grading; (5) students believe 
that working for competitively assigned 1-7 grades 
can be tension inducing, but such grading can also 
stimulate the student to work harder in his courses. 


foi hase: 

ве 2; The Academie Perfo ance P Lr" 

1 i he properties and assess 

Trying to discover the prop й ite 
worth of the pass-fail system by tapping student Ор! 
ions is not sufficient; one must also examine the 5у5- 
tem by studying undergraduate academic performance 
as a function of differential grading systems. There 
are two ways in which this task can be accomplished: 
(1) comparisons of academic performance between 


SOME GENERAL OBSERVATIONS BY PRINCETON UNIVERSITY UNDERGRADUATES CONCERNING 


THE PASS-FAIL SYSTEM * 


Multiple Choice Questions 


Ф Responding 


Question 
The best area to take pass-fail courses in is: 
15 
8 (à) math and natural Sciences 
30 (b) engineering 
(c) liberal arts and social sciences 
The best feature of a pass-fail course is it: 
i (a) lightens your workload 
can be treated as an *'intellectual ex rience" igi 
rather 
i m learning and reward A DER 
c) allows you to explore areas you mi i 
у ight have otherwise hesit: t 
12 (d) provides time to learn more in other courses НЕ 
(e) gets rid of grade pressures 
True-False Questions 
Question 
True False . No Opinion 
1f 1 couldn't have taken my 
pass-fail course on = 
basis, I wouldn't have taken it at all. AP PR. 26 68 6 
Right now, if I had m; i d 
y choice, Га change my pass-fai 
course to a 1-7 numerical grade credit, z2 € 
The j : i | 
Т е Бра matter of my pass-fail Course was more diíficult 
erial in most of the courses I take for a (1-7) grade, 31 65 4 


* All answers are given in 


ples (N-1,006). 


тсе: 
Percentages, and are based on the combined random and mailed questionnaire sam- 
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TABLE 7 
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MEANS AND STANDARD DEVIATIONS FOR THE TOTAL SUBJECT SAMPLE 


Variable x sD N 

Number of courses taken during the Academic Year 8.94 1.45 2,986 
Grade Point Average for the Academic Year 2.43 . 69 2,975 
SAT Verbal Score 646 67. 2 2,925 
SAT Math Score 691 65.8 2,916 


individuals who chose to exercise their pass-fail op- 

tion and those who did not; (2) comparisons of aca- 
demic performance by pass-fail students in their 
pass-fail and competitively graded courses, 


For the purpose of baseline comparisons, Ta- 
ble 7 presents the means and standard deviations for 
relevant academic variables based on the total sub- 
ject sample. Although most of these students 
(N 22,4154.) chose to exercise their pass-failoption, 
а small number (N=425+)didnot. Table 8 presents 


a comparison of means and standard deviations for 
these pass-fail and non-pass-fail students on rele- 


vant academic measures. The results pointto an im- 
portant finding: there is no significant difference be- 
tween yearly grade point averages for pass-fail and 
non-pass-fail students (F -2.07, NS). This was de- 
termined by calculating ayearly grade point average 
for non-pass-fail students (basedontheir grades in 
all courses for the academic year)andcomparing it 
with the yearly grade point average for yass-tall stu- 
Gents (calculated on the basis of performance in all 


courses except the pass-fail class). li, as theques- 
tionnaire data su 


Egest, pass-fail students gain addi- 
tional time for study in other courses by exercising 
their pass-fail option, then the above results indicate 
that this extra time did not lead to Superior perform- 
ance over those students who had no such extra time. 


TABLE 8 


С 


А COMPARISON OF MEANS AND STANDARD DEVIATIONS FOR PASS-FAIL AND NON-PASS-FAILS 


A second finding of importance supports the ex- 
pectation (t= Pvt < . 00001) that students will receive 
better grades in courses graded competitively than in 

those marked on а pass-fail basis. This was deter- 
mined by recording each pass-fail student's numeri- 
cal grade from instructors’ grade sheets before it 
was changed to a ‘“‘pass’’ (5-or better) or “ fail” 
(6 or 7) bytheregistrar'soffice. Thisgrade was 
then utilized for comparison with numerical grades 
achieved by the student in his competitively graded 
courses. The large differences in academic perform- 
ance as a function of differential grading systems can 


be noted by comparing the mean pass-fail grade 
(3.25 


) for all pass-fail students with their mean 
grade point average for the year (2. 42), в 


Thi 
finding supports questionnaire data where respond- 
ents indicated they worked harder for grades in com- 
petitively marked (1-7) courses. 


Based on academic performance data, one can 
wake the following summary statements concerning 
the Princeton pass-fail systems (I) students do 


equally well on their yearly grade point average it- 
regardless of whether t i 
fail option; (2) Xe they exercise their pass- 


р significantly bette 
in their competitively graded co ers 
os er eg courses than in their 


TUDENTS 


EMEN MM im U 
Pass-Fail Students 


Variable x 


Non- Pass-Fail Students 
x sD N x S5 x 
Number of Courses taken 
demic year 8. 
during the academic y 95 1.44 2,535 8.92 1.51 448 
Pass-Fail grade 3.25 1.05 2, 486 * i 
* 
Grade Point Average for 
the academic year 2. 42 .67 2,525 2.41 
ы „15 
SAT Verbal Score 644 66.7 2,490 655 ae 
i 69, 4 
SAT Math Score 690 66,2 2,484 695 E 
, 
63.2 
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DISCUSSION 


The purpose of this study was to examine student 
performance in course work, and attitudes toward 
such work, asa function of grading systems that dif- 
fered in their properties ( competitive vs. less 
competitive) and in such a way provide some empiri- 
cal information on the relative merits of the two mark- 
ing systems. The resultsof this investigation indicate 
that the pass-fail grading alternative is not the pana- 
cea that some educators envisioned. This is not to 
imply that a pass-fail system is worthless, only that 
findings reported here underscore the need for further 
Study and reexamination of the pass-fail system 
before it becomes a permanent grading procedure in 
our Universities. The urgency of this need is indi- 
cated in the rather unexpected responses of many 
students who often believed they learned more, worked 
closer to their capacity were more motivated to learn 
and more actively participated in numerically graded 
Courses than ones marked pass-fail. These beliefs 
were reflected in actual performance: students re- 
ceived significantly better grades intheir competitive- 
ly graded courses than intheir pass-fail subjects. X, 
indeed, students believe they learn more in competi- 
tively graded courses, one might ask why a pass-fail 
grading alternative should exist at all. Several an- 
swers could be given to this question (based on ex- 
perimental findings) including: (1) a willingness on 
the part of some students to explore courses they 
would ignore if faced with a 1-7 grade; (2) a reduc- 
tion in tension due to the elimination of competitive 
grading. Findings from the study also suggest that 
the effectiveness of the pass-fail system is under- 
mined when a student’s course load creates time pres- 
sures in his study schedule. When time is at sucha 
premium, pass-fail courses must compete with num- 
erically graded courses for a student's attention-- 
and under these conditions it is most likely that a stu- 
dent will withdraw time from his pass-fail class and 

invest it in the''dividend- paying'' numerically marked 
course. This is especially true at a school like 
Princeton where the undergraduates are accustomed 
to competitive grading and also have a high level of 
achievement motivation. In an academic system 


where all courses are marked pass-fail or where 


students have sufficient time to devote themselves 
in their pass-fail studies a pass-fail gradingalterna- 
tive will probably come much closer to fulfilling the 
goals for which it was implemented. 


FOOTNOTES 


l. This research was supported by an NSF Institu- 
tional Grant awarded to the investigator by 
Princeton University. 


2. The author gratefully acknowledges the generous 
assistance of thefollowing individuals: 
Gene Bruskin, John Hayes, Jim Kelly, and 
Tom Madeira for their aid inthe questionnaire 
phase of the investigation; Sandra Bullfor her 
data recording and processing; Thomas 
Coffman, Bertram Koslin and Richard 
Pargament for their assistance in data analy- 
Sis and, finally, to the Princeton student body 
for their aid in the project. 


3. In addition to providing a check for respondent 
honesty, the personalinterview questionnaire 
subjects served as a random sample of the 
pass-fail student population. Such a random 
sample was carefully selected and tested in 
anticipation that the mailed-questionnaire re- 
spondents might not represent an accurate 
cross-section of all pass-fail students. 


4. This figure represents just under a 50 percent 
return on questionnaires mailed out. 


5. Questions were not given to students in the order 
appearing in the Table. See Appendix A for 
actual order of presentation. 


6. Note how closely theactual mean pass-failgrade 
(3. 25) corresponds to the Students mean es- 


timation (Table 5) of what their pass-fail 
grade will be (3. 22). 


APPENDIX A 
PASS-FAIL QUESTIONNAIRE 
Section 1, 
True-False. In this section please circle a “T” і 
think it is False; circle “0” if you have no Opinion conce 
questions based on your experience with this sem 
T True F False 
Т Е 0 (1) AsfarasI can determine 


РЕ 1 


CE— 


d Hd HA 
"uj "jon ou 


d 
"n 


4 
" 


B48 a 4 
"nj "j ы эы 


F 


F 


F 
F 


d H HH HH 3 H H A 


hj jo y y 


0 (5) 
0 (6) 
о (7 
о (8) 


0 (9) 


0 (10) 
0 (11) 


0 (12) 
0 3) 
0 (14) 
0 (15) 
0 (16) 


(17-24) 


0 (17) 
0 (18) 
0 (19) 
0 (20) 
0 (21) 
0 (22) 
0 (23) 
0 (24) 


Section 2. 


(25) 


(26) 


(27) 
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Right now, if I had my choice, I'd change my pass-íail course to 1-6 numerical grade credit. 
If I couldn't have taken my pass-fail course on a pass-fail basis I wouldn't have taken it at all. 
More courses should be graded on a pass-fail basis. 
The pass-íail system should be abolished at Princeton. 


I devote as much time to my pass-fail course now as Ithought I would when I signedup for the 
course. 


The competition for grades in courses stimulates me to work harder. 


The subject matter of my pass-fail course was more difficult than material in most of the courses 
Itake fora (1-6) grade. 


If I had more time I'd work harder in my pass-íail course. 

If I knew my course was being graded 1-6 rather than pass-fail, I would work harder. 

I didn't explore my pass-fail coursework fully because I knew it wouldn't affect my grade. 
I have tried to do as well gradewise in my pass-fail course as in my numerically graded courses, 
Iwas able to learn more in my pass-fail course because it was not graded on a 1-6 basis. 
If I was taking my pass-fail course for a normal 1-6 grade I would: 

Do more of the assigned readings. 

Attend more of the lectures and/or precepts 

Study harder for course quizzes and exams 

Do better work on papers, labs , etc. 

Be more interested in the course in general 

Be under more tension due to worry over grades 


Gain more knowledge 


Gain more intellectual stimulation 


Multiple choice. In this section please place the number of the alternative th: 
sentiment concerning the question to the left of that question number. 
per question, Please answer these questions based on your experienc 


at best expresses your 
Choose only one alternative 
€ with this semester's Pass- 

———EDester's Pass: 


The best feature of a pass-fail course is it: 

(1) Lightens your workload. 

(2) Can be treated as an “intellectual experience” 
and reward. 

(3) Allows you to explore areas you might have otherwise hesitated to delve into. 

(4) Provides time to learn more in other courses, 

(5) Gets rid of grade pressures. 


rather than a rigid process of learning 


Iam learning: (1) much more 
from my pass-fail course. 


The worst feature of a pass-fail course is: 
(1) It is often wasted since I tend 
(2) I still worry about passing it when I don't do all the песе: 
(3) It's an option I wouldn't exercise except that ever 


(4) The fear of getting a “good grade” for nothing. 
(5) None of the above. 
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(28) I feel that I work closer to my capacity: 


(1) Ina numerically graded course rather than one graded pass-fail 
(2) In a pass-fail course rather than one graded numerically 


(3) About equally in both types of courses 


(29) In my pass-íail course (in comparison to numerically graded courses) I have had: 
(1) an increase in motivation to learn 
(2) a decrease in motivation to learn 
(3) no change in motivation to learn 


-— о. 


(30) As far as the amount of material learned is concerned, I have learned: 


(1) much more (2) more (3) the same (4) less (5) much less, in my pass-failthana compara- 
ble numerically graded course. 


(31) The best area to take pass-fail courses in is: 
(1) math and natural Sciences ac 
(2) engineering j 
(3) liberal arts and social sciences 
| 


(32) Which of the following factors was most important in your choice of a pass-fail course ? 
(1) the difficulty of getting a good numerical grade : | | 
(2) the subject matter of the course was unfamiliar and/or more difficult than in other courses 
taken for numerical credit 
(3) the subject matter of the course was familiar and/or less difficultthan in other courses taken 
for numerical credit 
(4) specific requirements of the course: 


(e.g., paper, exams, etc.). 
(5) none of the above 


Section 3. 


Miscellaneous. Please fill in each question according to directions, Please answer these ques- 
tions based on your experience with this semester's Pass-Fail course 


(33) Considering the amount of work done in your pass-fail course, what numerical grade would you 


| 
expect ? А е | 7 
What kind of grade would you aim for if you were taking the pass-fail course for credit ? Я \ 


(34) Below are three possible goals that might be achieved in giving pass-fail courses at Princeton | 
(you can use a goal more than once in answering the questions): 
(1) To reduce the tension of, and emphasis On, course grades. А |. 
(2) To provide additional study time for other courses and/or extra-curricular АСНУ Нев; | 
(3) то encourage course selection on the basis of interest rather than grade considerations, 


=_=. In the space to the left write the number of the goal that you think best describes the University's 
reason for implementing the pass-fail system. 


= In the space to the left write in the number of the goalthat you think best describes your reason 
for taking a course on a pass-fail basis. 


CON IN In the space to the left write in the number of the goal that you think best describes the reason why 
most students at Princeton take a course pass-fail. 


(35) In the blank that follows writ. 


Semester - Then please circle the numbers below that 
compare your pass-fail cour 


Semester, my pass-fail course has been the Ist (most), 2nd, 3rd, 4th, 5th, 
ing (i (NOTE: If you took four courses and 
y stimulating you should have circleda ''1st" above; if it 
was the second most stimulating course, а **2nd" above; if it was least stimulating of the 4 courses 


a “А” above, ete. If you had taken five courses instead of four and the pass-fail was least stimu- 
lating of these five, you should have circled a ‘‘5th’’ above. ) 


(b) Of my courses this sem 


ester, I have learned the 1. 
amount of information i 


St (most), 2nd, 3rd, 4th, 5th, 6th ( least ) 
n my pass-fail course. Ы 
(с) оғ my courses this semester the subject matter of m Ж 
y pass-fail course has been the 1st ( most 5 
and, 3rd, 4th, Sth, 6th (least) difficult, ' ; 
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(d) Of my courses this semester, I studied 1st (most), 2nd, 3rd, 4th, 5th, 6th, (least) in my pass- 
fail course. 


(e) Of my courses this semester, my motivation to learn was 1st (most), 2nd, 3rd, 4th, 5th, 6th 
(least) in my pass-fail course. 


(36) Utilize the percentages listed below in answering this final set of questions about your 
semester pass-fail course. In each blank place the percentage figure that most close 
sents the true amount of effort you invested in a specified activity. 


Spring 


ly repre- 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 


(a) In a numerically graded (1-6) course I do % of the assigned readings; in my pass-fail 
course I do % 
(b) In a numerically graded course I attend 


% of the lectures and/or precepts; in my pass- 
fail course I attend 7. 


(c) Ina numerically graded course I work to my capacity 


% of the time; in my pass-fail 
course I do so % of the time. 


(d) In a numerically graded course I am interested % of the time; in my pass-fail course I am 
interested % of the time. 


% of the material presented;in my pass-fail course 


(е) In a numerically graded course I learn 
Ilearn Ф . 


Thank you for your cooperation. 


APPENDIX B 


COURSE COMPARISON QUESTIONNAIRE 


INSTRUCTIONS: 


We are interested in finding out what you think about your current spring semestercourses, Here 

is a list of those courses. We would like you to compare these courses by rating them from 
**most"' to **least"' on certain educational attributes, Forexample, we want to know how you rate 

Thus, for this aspect of your courses, 
is se- 
ually stimulating, and so on down to the 
the ten items below.In- 


clude all your courses in each rating and please do not permit any ‘‘ties’ between Courses in your 


a 


Please list below the courses you took in order of how 


intellectually stimulating (interesting) they 
were: 


Most intellectually stimulating Least intellectually stimulating 


(2) Please list below the courses you took in order of the amount of information learned in them: 


Most information learned Least information learned 


(3) Please list below the courses you took in order of how difficult the subject matter was in th 
em: 


Most difficult subject matter Least ditficult subject intem 


(4) Please list below the courses you took in order of how hard you studied in them: 


Studied Most Studied Least 


50 THE JOURNAL OF EXPERIMENTAL EDUCATION 


(5) Please list below the courses you took in order of how motivated you were to learn in them: 


Most motivated to learn 


Least motivated to learn 


(6) Please list below the courses you took in order of how much grade pressure there was: 


Most grade pressure 


(7) Please list below the Courses you took in order of how many lectures and/or precepts youatte 


Most lectures/precepts attended 


(8) Please list below the courses you took in order of how much of the 


Most assigned reading done 


(8) Please list below the c 
ty in them: 


ic Worked closest to capacity 


ourses you took in order of how close you c 


Least grade pressure 


nded: 


Least lectures/precepts attended 


assigned reading you did: 


Least assigned reading done 


ame to workingupto your capaci- 


Worked least closest to capacity 


(10) Please list below the courses you took in order of how hard you worked for grades in them: 


Tried hardest for grade 
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PERCEPTUAL-COGNITIVE STYLE AS RELATED 
TO SELF-EVALUATION AND SUPERVISOR 


RATING BY STUDENT TEACHERS 


DONALD MUSELLA 


Ontario Institute for Studies in Education 


ABSTRACT 


Research in the area of social perception has pointed to (1) therelationship between certain personalchar- 


acteristics and the perception and cognition of others, 

ments of others, and (3) the influence of dogmatism on 
others. An earlier study by this researcher provided e 
tion and rating of elementary school teachers by eleme 


(2) the effects of certain personal characteristics on judg- 
the perceptual-cognitive style affecting judgments of 
vidence supporting the effect of dogmatism on the descrip- 
ntary school principals. This study represents an effort 


to provide additional information related to (1) the relationship between dogmatism and self-evaluation in teach- 
ing, and (2) the relationship between dogmatism and the evaluation of supervising personnel. A review of litera- 


ture and of previous work in this area by the researcher 


produced hypotheses related to differences between open- 


minded and closedminded student teachers and their rating and description of themselves and of their supervising 


teachers. The results support the hypotheses in this study, 


as well as the results of the previous study. These 


results indicate that the rating of one’s teaching, which may be considered a manifestation of the evaluation of 


self in total, and the rating of one’s superordinates are 


style of the rater. 


RESEARCH IN the area of social perception has 
pointed to the relationship between certain personal 
characteristics and the perception and cognition of 
others ( 1, 2, 3, 4, 7, 8, 10). Severalstudies have 
identified the effects of certain personal characteris- 
tics on judgments of others (6,9, 14, 21, 22, 23, 27, 28). 
Other studies have dealt more specifically with the 
influence of dogmatism on the perceptual-cognitive 
style affecting judgments of others (5,12, 13, 15,16 
19, 21, 23, 25, 29), ы 


An earlier study by this researcher provided evi- 
dence supporting the effect of dogmatism on the des- 
cription and rating of elementary school teachers by 
elementary school principals (16). 


The present study represents an effortto provide 
additional information related to 1) the relationship 
between dogmatism and self-evaluation in teaching 
and 2) the relationship between dogmatism and the 
evaluation of supervising personnel, 


While the results of the earlier study were clear 
and unambigious, the limitation of sample (elemen- 
tary teachers and administrators) raises questions 
about the generality ofthefindings, The present stu- 
dy constitutes a partialreplication, aimed primarily 
at determining whether the interrelationships emerge 
under different conditions, with different populations, 


› in some respects, a function of the perceptual-cognitive 


and with an additional supportive measurement de- 
vice. 


HYPOTHESES 


A review of literature and of the previous w ork 


in this area by this researcher produced the following 
hypotheses: 


(1) closed- minded student teachers would tend 
to rate themselves higher in teaching than 
would open- minded student teachers; 


(2) closed- minded student te 
achers woul 
to rate supervising teachers lower re А 


ing than would -mi 
in open-minded student teach- 


(3) closed-minded stud 


(4) closed- minded student teachers would tend 


ity in their descriptions of 
would open-minded student 


Uv 
N 


oups of student teachers would tend 
(5) керы Tdentieal characteristics consid- 
алеп essential for effective teaching. 


METHODS 


Sample 


d of 128 student teachers 

T Cup AES кыже College at Cortland, 
poe dii Each student teacher was assigned for 
| ewe Seweek period to one of sixteen supervising teach- 
ers in the Campus- Laboratory School. In most cases, 
the student teachers were selected by the supervising 
teachers on the basis of the initial teac hing experience 
provided for eachcollege student during the previous 
year. These students were considered, by the super- 
vising teachers, to be potentially above-average stu- 

dent teachers, 


The total sample consisted of thirty-two male 
and ninety-six female students who were enrolled in 
the elementary education program during their jun- 
ior year of college. The Supervising teacher group 
was comprised of five male and elevenfe male teach - 
ers of grades nursery through 6. All supervising 
teachers were full-time professional staff assigned 
to the laboratory school. 


From this sample, two groups were identified: 
the upper 25 percent scoring high in dogmatism 
(closed- minded student teacher -- CST), and the 
lower 25 percent Scoring low in dogmatism (open - 
minded student teacher -- OST). 


Instruments 
Tsuuments 


loped by Rokeach and his 


the D Scale (closed- minded sub- 
o ring low on the Scale (open- minded 
ed them to various tests, prob- 


Inthe samples used, the 
тот . 68 to , 93 


The TPCI, developed 


pairs of polar terms, arrived at by 


е, which rep- 
E and the least 
rS. ese characteristics were de- 
rived from Over 500 critical incidents submitted by 
Кеней administrators, teachers » practice - 
eachers, and college teachers, 


The TEF, develo; 
а modification of th 
and others (20), т 


S of the most effective 


реа by this researcher (7), is 
e rating scale developed by Riley, 


erical expression of one 
In addition, most ef- 
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TABLE 1 


COMPARISON OF SCORES BETWEEN MALE 
AND FEMALE STUDENT TEACHERS 


Male Female Total 

Number 32 96 128 

D Scale 
Range 81-189 90-195 81-195 
Mean 141.7 140. 8 144. 4 
S.D. 24.1 22.5 21,7 
not significant 

TPCI 
rr 
Range 82-130 88-158 82-158 
Mean 122,4 120.3 124.2 
S.D. 23.6 21.9 22.1 
not significant 

TEF, 
RÀ т. 
Range 29-86 31-82 29-86 
Mean 62.7 60.0 61.1 
S.D. 8.1 9.5 9.2 
not significant 

TEF 

sp 

Range 41-80 45-90 41-90 
Mean 70.5 72.9 12.1 
S.D. 8.4 9.2 9.3 


not significant 


not significant 


*Accumulative Aver: 
less than 2.0 
not permitted 


age Grade Point. Students with 
at the end of the Sophomore year were 
to student teach (department policy ). 


Procedure 
== cedure 


Student teachers were аѕѕі 
gned to the Campus- 
Laboratory School for a period of 8-weeks. The Spe- 


Scale. After the conclusion of thi 
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experience, and after the supervisor had submitted 
final grades, each student teacher was asked to 

1) rate the supervising teacher on teaching (not 

supervision) using the TEF, 2) rate himself/her- 
self on teaching on the identical Scale--TEF, 3) de- 
scribe himself/herself on the TPCI, and 4) select А 
from the list of positive traits on the TPCI, the five 
characteristics most essential to effective teaching. 


The controls inherent within the Study were as 
follows: 


(1) all student teachers spent a relatively equal 
amount of time in immediate contact with each 
supervisor; the situation was such that both 
the supervisor and student taught the same 
class every day;each had ample opportunity to 
observe each other's teaching in the self- 
contained classroom setting; 


(2) all student teachers had similar formaleduca- 


tional experiences; all were elementary educa- 
tion majors in their 3rd year at the same col- 


lege; 


(3) all assignments were based on the choice of 
both the student teacher and the supervising 
teacher; 


(4) none of the student teachers hadany previous 
formal teaching experience; and 


(5) there were no significant differences between 
the male and female student teachers on the 
accumulative average grade point at ће end of 
the previous academic year. (See Table 1.) 


TABLE 2 
COMPARISON OF TER, SCORES BETWEEN 
CST AND OST 


CST OST 
Range 39-86 31-82 
Mean 71.3 62.4 
S.D. 7.6 8.2 


n 32 32 
t=1, 99; significant аё. 05 


COMPARISON OF ТЕР, SCORES BETWEEN 


CST AND OST 


TABLE 3 
COMPARISON OF TPCI SCORES BETWEEN 


CST AND OST 


CST OST 
Range 105-158 91-158 
Mean 140, 4 129. 6 
S.D. 15.0 15.3 
d. f. 31 31 


t -3. 87; significant at . 01 level 


Item Scores 


Range 3-6 2-6 
Mean 5. 71 6. 29 
S.D. .27 "unl 
Var. .081 1.152 
d.d. 31 31 


Е -10.39; significant at.01* level 


RESULTS 


A summary of data received from the four Scales-- 
D Scale, TPCI, TEF, (self), and TEF sy (supervising 


teacher), indicate no significant differences between 
male and female students on any of the scales, (See 
Table 1.) Therefore, the total student teacher group 
can be considered a relatively homogeneous sample 
in terms of sex differences related to behavior on the 
Scales and to success in college, 


The statistical results, as provided in Tables 2 
3, and 4, are summarized as follows: ü 
(1) elosed- minded Student teachers (CST) ratea 
Selves higher on teaching effectiveness than 


did open- mind: 
a - minded student teachers (OST) (Ta- 


(2) CST rated Supervisors lower on 
tiveness than did OST (Table 2) 


(3) CST described themselves i v 
ч n mor e iti 
к t Te positive 
vp (high rating Scores) than did OST (Ta- 


teaching effec- 


(4) CST displayed less variability į 
: ariabil i i 
tions of self than did OST (Table din Sohal. 


(5) CST ana OST did not differ si 


CST OST 
Range 41-82 51-86 
Mean 62.3 79.5 
S.D. 8.4 8.9 
й 32 32 


t-2. 23; significant at . 05 


= ee ee 


, and (e) fair; 
oe (c) sti: 
1 А €) unders; i 

relation between groups was nd, 


(See Table 4, ) Positive 


1 


Ls 


e 
min стык nam < 


a 
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TABLE 4 


COMPARISON OF CHARAC 


TERISTICS SELECTED 
BY CST AND OST 


Characteristics No. CST No. OST 
NE ————— — . 09099 
Stimulating 6 5 
responsible 5 T 
Systematic 5 H 
contident 4 5 
fair 4 2 
democratic 2 1 
understanding 2 4 
active 1 1 
adaptable 1 5 
optimistic 1 0 
original 1 1 
permissive 1 0 
responsive 1 0 
alert 0 0 
altruistic 0 0 
attractive 0 0 
broad 0 0 
integrated 0 0 
kindly 0 0 
poised 0 0 
positive 0 0 
relative 0 0 
Steady 0 0 


г=,91 (Ѕреагтап rho) 


DISC USSION, CONC LUSIONS, ANDI MPLICATIONS 
The resul 


Sults cou] 
retical development of Rokeach’ 
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findings lend 
they suggested 
istics of the rater are related 
'S self and of 
the evidence 
es are discerned between pre- 
predominantly closed student 
ng and discription of teaching 
rse, itis possible that the dif- 
ferences could be the result of differences in indi- 
viduals, especially since the subjects had no previous 
teaching experience u pon which to base judgments. 
However, similar results were found with experienced 
principals and teachers (16). Furthermore, it was 
found that differences existed between open and closed 
Student teachers in the description of self on the 
TPCI. Closed subjects displayed less differentia- 
tion and variability in their descriptions. They 
tended to see themselves in completely positive 
terms, whereas the open subjects pointed to both 


positive and negative characteristics in their des- 
criptions of self, 


No significant differences w 
tion of characteristics с 
sary for effective teachi; 


ing, one might Speculate tha 


The implications of thi 
vance to existing theories 
style and to theories which 
logical orientation to inter 
pecially interpersonal perception) in a social and 
organizational context, The importance of personal 


characteristics becames apparent in the examination 
of other forms of decision- i 


is study bear direct rele- 
of perceptual-cognitive 
employ a Social-psycho- 
personal relations (es- 


The limitations of this study are readily apparent 
in terms of population size, population characteris- 
tics, measurement devices, and the quantity and 
quality of interpersonal contact, 

sults must e 


variables, espe 
sonal contact, [n addition, 
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LEARNING OUTCOMES 
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DALVA E, HEDLUND 
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ABSTRACT 


sistent improvement in 
that the 


Psychology class at CSU (N-297); 


mester Introductory Psychology Class at the Univer- 
Sity of Missouri (N-185). АП thr 


i taught by closed-circuit television lectures, and all 
poi i classes met in sections of approximately th irty to 
T: y cational Objections: Affective Do- forty students taught by graduate assi 
main (3) that the implicit assumption behind most 
teaching efforts 


lecture Series Presented to h 


both CSU classes was on 
Ognitive Outcomes of tape and thus Virtually identical between classes, ex- 
a ty E Xperience achieved there will automati- cept that the Fall class had a few extra tapes, 
cally ea COrrespondin development of appropriat 
affective behaviors. D Г JESUS 


Ti 
owever, there is little, if ап; 
empirical data оп the r, 


А Attitude Toward Psychology Scale 
elationshipover time of affec- 
tive and Cognitive learning Outcomes, 


To develop an attitude tow. 
The present Study was de. 


a taxonomy Of areas in which attitudes were probably 
Signed to investigate the instructional objectives was developed from an ex- 
longitudina] Covariation of attitude toward the subject amination of several widely used Psychology text- 
matter (the affective outcome) and achievement (the ooks, Statements were then drawn from CSU text 
Cognitive Outcome) during а Course of instruction, and taped lectures to confi i 
METHOD 


a ited for clarity and brevity. These 
Statements were th 
Situation and Subject ates 


mitted to eight judges, three 
wie асу members, and five graduate assistants 
: who ha experience with Introductory Ps cholo 
The longitudinal design of study required repeated e instructors is 
ficte qu attitude toward the subject matter (af- 
Ctive outei 


9 discourage the attitude, or Were 
ка на "Rn nd t in class, Only items on which al] 
me times durin a Judges agree, rfecth - 
Course of instruction, Introductory Psychology was aging were conside ce" Inclusion yur - 
duin as the subject Since large numbers of stu- tude scale, Те items representing the attitude m 
m ae Ww ed е cooperation of instruc- onomy were chosen for the fina] attitude sc. le. Ont 
as mosi easily be Obta; Thr. € Separate ten items were Chosen 9 minimize administr ti 
Ses of instruction Were involved: the Fall, 1966 , time while assuring the Possibility or btaining sap. 
MN y Psyc logy class atColorago State Unj- isfactory reliability, TI ems were set Likes 
ty (CSU) (N-384); the. Winter, 1967, Introductory 5-point Constant response abn 


——— 
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0-4. Scores on the attitude scale were the sums of 
the weighted responses to the ten items. 


It was decided to measure attitude toward psycho- 
logy and achievement after each successive 20 per- 
cent of a course of instruction. To accomplish the 
repeated measurements, five equivalentforms of the 
attitude scale were developed by changing unimpor- 
tant words in attitude statements and randomly rear- 
ranging items within the scale. The discrimination 
power of eachitem was checked on a random sample 
of one hundred subjects drawn from the CSU Fall 
class and divided into upper and lower 50 percent 
groups on the basis of totalattitude score, Split-half 
reliability coefficient, corrected by Spear man- Brown 
prophecy formula, were calculated for all five 
forms of the attitude scale from from data on this same 
one hundred Ss. The corrected coefficients ranged 
from .35 to . 83, with most at about . 60. Correla- 
tion coefficients between adjacent administrations of 
the attitude scales were also calculated in all three 
classes, yielding an estimate of test-retest reliabili- 
ty over a period of 2 to 3 weeks between different 
forms of the attitude scale. The fifteen test-retest 
reliability coefficients ranged from . 56 to. 83, with 
all but three coefficients above . 66. 


Achievement Tests 


Five successive achievement measures were also 
necessary to detect changes in achievement during a 
course of instruction, A pool of approximately 500 
objective achievement items was constructed from an 
analysis of the CSU textbook and TV lectures, Items 
in this pool were administered to seventy-seven stu- 
dents enrolled in a summer session Introductory Psy- 
chology course. Items which successfully differenti- 
ated high from low achievers inthis pretestseries 
were maintained in the item pool. Since the primary 
purpose of a repeated achievement measure was to 
reliably differentiate students on the basis of c lass 
achievement or cognitive learning, a traditional learn- 
ing curve with equal-intervalincrements was not nec- 
essary. For this reason, no attempt was made to 
equate difficulty levels or content of achievement 
items. It was arbitrarily decided to cover each 20 
percent portion of a course withfive achievement items. 
Moreover, it was reasonedthat items covering those 
parts of a course which had not yet been presented 
would not be answered correctly. Therefore, only 
items based on topics previously presented inclass 
were administered at any time. — Following this pro- 
cedure, five achievement items were added to the 
achievement test following each 20 percent of the 
course. The final achievement test included twenty- 
five items, five of which covered the topics presented 
in the final 20 percent of the class, and twenty of which 
covered all topics presented previous to the final 20 
percent of the class. A similar pattern was followed 
for each of the five achievement tests: five new items 
were added to cover the immediately preceding por - 
tion of the course, and five other items were chosen 
to cover other material presented in each preceding 
20 percent portion of the course. The first achieve- 
ment test was based only on standardized scores ob- 
tained from instructor-constructed quizzes. The fac- 
ulty coordinator for each course chose items which 
were representative of material presented in class 
for each succeeding achievement test from the estab- 
lished item pool. Thus, the number of items at each 


successive administration was five (based on standard- 
ized score), 10, 15, 20, and 25. In each case,items 
were representative of topics presented in class to 
that point in the class. Achievement scores were the 
number of correct responses. 


Content validity of achievement measures was as- 
sured by the method of item construction and selec- 
tion. Thecorrected split- half reliability coefficients 
for the last four administrations based on data from 
the previously described random sample of one hun- 
dred students reflected the increasing number of 
items through successive measurements. These re- 
liabilities ranged from. 36 for the second administra- 
tion to . 75 for the fifth administration. A corrected 
split-half reliability coefficient of . 63 was also ob- 
tained on the final achievement measure for a ran- 
dom sample of twenty-five students from the Missouri 
class. 


It should be noted that in the construction of the 
attitude toward psychology scale andthe achievement 
items the Missouri instructors concurred that the 
items were also objectives of their course of instruc- 
tion. 


Data Collection 


A prepared statement introducing the study was 
read to students by section instructors at the first 
administration of the attitude scale. Attitude scales 
were administered by section instructors at approxi- 
mately the end of each 20 percent of the respective 
courses. The order of administration of the five 
forms of the attitude toward psychology scale was 
the same for all classes, Since items within the 
scales were randomly arranged, counterbalancing 
of forms did not seem necessary. To assure that 
student responses would not be biased by the possi- 
bility of instructors examining the completed Scales, 
completed scales were returnedto a student monitor 
who sealed all scales for his section in an envelope, 


Achievement measures were obtained either as 
part of a regular classquiz administered in Section 
meetings or as keyed items on regular one-hour ex- 
aminations. Inclusion of achievement items in the 
context of regular class testing assured that students 
would respond to the best of their ability, In all 
Cases, achievement items were drawn from the es- 
tablished item роо]. However, the items at any ad- 
ministration were not necessarily the same between 
class situations, nor were the items necessarily the 


same for any class topic over administrati 
in a given class, eee 


Analysis of the Data 


Curves were fitted to the attitudinal i 
achievement variable over time, The eive Gane 
was accomplished by determining the mathematica] 
function which most adequately accounted for th ў 
tal variation in the variable through a polynomial Te 
gression analysis (5). Ву fitting curves to attitude | 
» it was possible 
S over time and 
ation was evident, 
The three Introductory Ps: 


treated as c omplete rep ychology courses were 


lications of the Study to test 


"S = © 
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the generalizability of the re. 

Sults. А11 statisti 
analyses were program Hae 
dod g.: med for a CDC 6400 comput- 


RESULTS 


In order to conserve s 
pace, the complete lysi 
will not be reported, The regression Сы рек 
sulting from the data for each variable in eachof the 
learning situations we 


re solved, and the resultin 
predicted scores converted to a ‘percentage of the 


| total possible score sothecurves cou 
the same coordinates for mehr a p 
and 3). Although the equations for each variable dif- 
fered somewhat from situation to situation, it will 
be noted that the resulting curves are of similar 
shape across learning situations. 


Achievement in all three situations rises in near- 
ly a linear manner, although the curves are repre- 
sented by as highasa fourth-degree equation, Scores 
on the Missouri final achievement examination were 

about 10 percent lower than scores on the two CSU 
final examinations. 


Attitude toward psychology was best described by 
a horizontal mean line in the CSU Winter and Missouri 
classes. A slightly rising linear function described 
the attitude change in the CSU Fall class. It is ri 
dent that, inthe three classes studied, рес ход 
in the learning experience did not change attitut 
toward subject matter. 


FIGURE 1. CSU FALL PREDICTED CURVES FOR ATTIT 


ACHIEVEMENT SCORES. 
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j There appears to beno relationship over time be- 
ween the two variables in any situation studied, 


TABLE 1 


MEAN ACHIEVEMENT SCORES BY ADMINIS 

A TRA- 
TION OF HIGH AND LOW FINAL ATTITUDE 
TOWARD PSYCHOLOGY GROUPS COMPARED 


WITH PREDICTED MEAN ACHIEVEMENT SCORES 
OF THE TOTAL CLASS 


Administration 
Group 1 2 3 4 5 
High (N-38) 2.66 8.97 12,55 16.00 20,23 
Predicted 2.62 8.16 12.44 16,12 19.88 
Low(N-38) 2.66 8.08 11.94 15.81 19.26 


Achievement and Changing Attitude 
Toward Psychology 


i i 5 hanges 

At this point it was decided to compare c 

in азаеп over time veg pim pii eru 
terns of change in attitude towar EY 

x the Winter CSUclassas subjects, those individ 

uals in the high and low 25 percent of the distribution 


UDE TOWARD PSYCHOLOGY, AND 


Achievement 
LL... Psychology 
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FIGURE 2, CSU WINTER PREDICTED CURVES FOR ATTITUDE TOWARD PSYCHOLOGY. 


AND ACHIEVE MENT SCORES. 
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of the fifth attitude toward psychology scores were 
identified. Within the high and low final attitude 
groups, the 50 percent of the students nearest the to- 
tal class mean on the first administration attitude 
toward psychology scores were identified. A group 
beginning near the class mean and exhibiting a rising 
attitude toward psychology (N-38) and a group begin- 
ning near the class mean and exhibiting a declining 
attitude toward psychology (N-38) were thus identi- 
fied. If affective were related to cognitive learning, 
the achievement curves for these groups should have 
differed from each other and from the curve of the 
total CSU Winter class. Mean achievement scores 
were calculated for each group and compared to 
the predicted achievement scores for the entire 
class. As can be seen from Table 1, the high, 
iow, and predicted achievement means could hardly 
be closer if completely random groups had been 
drawn. There isa slight trend for the high attitude 
group to have higher achievement scores than the 
low attitude group. However, the differences are 
not large enough to indicate any realrelationship be- 
tween affective and cognitive learning when the 
method of choosing the groups is considered. 


DISCUSSION AND CONCLUSION 


It appears that in the three learning situations 
studied that the cognitive goals, measured by objective 
achievement items, were attained, At least there 


was sufficient evidence from the resulting achieve- 
ment curves to conclude that performance was con- 
siderably better at the end of each class than at the 
beginning. Affective teaching objectives, however 
did not appear to be attained. Attitude toward ps; M 
chology, measured by attitude items that all =. ий 
(отв Bessey ву were trying to influence, was, at 
est, merely maintained thr i 
Rete ет y oughout a course ofin- 


Contrary to the most common assum 
the relationship between cognitive and x aee 
ing outcomes, it appearsthatthe two realms are = 
atively independent. Evidence from this stud: st Long 
ly suggests that changes in one realm do nd stan 
sarily lead to changes in the other realm. "Hi coy 
clusion was supported by the similarity of the cate 
in all three learning situations. Further su cio gus 
given to the conclusion of independence by Tp a e 
siete students with diverging attitudes toward s us 
hone eee not differ in cognitive doble Enn 
predicted achievement of the total clam, 


The use of on 
television could hes ate 
хады goals are more difficult to teach 
MÀ epe ised presentation or that the liek 
раа ructor-student interaction maki = 
olvement with the subject matter [wena 
о 


ght by closed-circui 
hese results, 1 mM 
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achieve. However, the individuals which were identi- 
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ABSTRACT 


This experiment compared three sequence versions ( logical, scrambled, and reverse frame orders ) of 
the guided discovery program, Number Series, used by Gagné and Brown in their 1961 study, Sixteen ninth- 
grade algebra students in each of the three sequence groups plus a control group served asthe subjects. While 
the logical order group was the only sequence group to perform significantly better than the controls on both a 
test of concepts and a problem solving test, none of the three sequence groups differed significantly from each 
other on posttest performance. Logical group students did, however, make significantly fewer program errors, 
They also tended to consider the program ''interesting,'' whereas scrambled and reverse order groups felt ‘‘neu- 


tral” about the program. 


THE NOTION of sequence has long been enshrined 
in the theory of curriculum in general and pro- 
grammed instruction in particular. Skinner, in his 
classic article, “Тһе Science of Learning and the 
Artof Teaching,'' indicated the logical necessity of 
sequence: ‘‘The device[teaching machine] makes 
it possible to present carefully designed materialin 
which one problem can depend upon the answer to the 
preceding and where, therefore, the most efficient 
progress to an eventually complex repertoire can be 
made’ (11:84). If the frames of a rather sophisti- 
cated unit of instruction were presented in a randomly 
sequenced manner, few people would doubt that the 
resultant learning would be less efficient and less 
effective than a carefully planned sequence where 
“each successive experience builds upon the preced- 
ing one. . ."' (12:55), 


Thus it is a rather perplexing state of affairs 
when experimenters, using programmed instruction 
to control sequence and content, are unable empiri- 
cally to validate the sequence hypothesis. At least 
six studies over the past 8 years (5, 6, 7, 8, 10, 13) 
have shown no significant differences in learner 
achievement when comparing logical and random se- 
quence treatments. In the one study that does sup- 
port the sequence hypothesis, Roe (9), the results 
are uncertain. Roe found a significant difference 
(10 percentage points) favoring a logical sequence 
when comparing logicaland scrambled programs ad- 
ministered on 4 x 6-inch cards in file boxes to col- 
lege engineering students. Yet another version of the 


logical program, administered by means of a pro- 
grammed text format, produceda meanthat was six 


percentage points lower thanthe scrambled sequence 
on the cards. 


From the remaining studies related to logical and 
random sequencing of programmed materials (i.e 


the “по difference” studies), thr 
пеле. ) ee hypotheses have 


1. Sequence effects depend upon the subj 
à ject mat- 
ter being taught, The more interrelated he 
Eee ee ee sense of a Gagné type hi- 
y (3)---the more effect s 
on learning. PES MAN 


2. Sequence effects de 
А ‹ pend upon the indivi 
learner's ability. High ability ieee P 


able to overcome the effects of poor sequence 


ing better than low ability learners, 
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that scrambled and reverse order Sequences would 
be less efficient and less effective ( as indicated by 
instructional time, error rate, and posttest perform- 
ance) than the logical sequence. It was also hypothe- 
sized that posttest differences between high and low 
IQ subjects would be greater under Scrambled and re- 
verse conditions than under logical order. 


METHOD 


Subjects 


Sixty-four ninth-grade students comprising t wo 
algebra I classes at a suburban junior high schoolin 
the Los Angeles area servedas subjects in the inves- 
tigation? They were randomly assigned to logical, 
scrambled or reverse order sequence conditions and 
а control group, making a sub-total of sixteen in each 
of the four groups. California Test of Mental Matu- 
rity (CTMM) Total 195 were available опа but four 
of the students, and posttest results were available 
оп all but one of the logical students who was absent 
on the day of the tests. The mean IQfor each group 
was between 111 and 112, and the variances within 
each group were not statistically different. 


Program 


This experiment used a modified version of the 
Number Series program? from Gagné and Brown's 
1961 study, “Some Factors in the Programming of 
Conceptual Learning" (4). The original program 
consisted of eighty-nine introductory frames followed 
by forty guided discovery frames which requiredthe 
learner to ‘‘actively produce, .. concepts...which 
he hadjust previously learned in the introductory 
program’? in order to attain the problem solving skill 
of *'finding a formula for the sum оГ п terms ina 
number series'' (4:319), This particular learning 
program was selected because it was thought to be 
based on the knowledge hierarchy validated for this 
Same terminal task by Gagné in his 1961 article ; 
“The Acquisition of Knowledge" (3). 


Initial tryout of the program with individuallearn- 
ers found them rapidly ‘‘extinguishing”’ on the eighty- 
nine introductory frames. Twenty-eight frames 
were then systematically removed. (For example, 
if the item, ‘The term-value for n=5is ," ap 
peared six times, with different numerals for n, then 
two of these practice frames would be removed. ) 
This modified, 61-frame version proved equally as 
effective as the 89-frame introductory program with 
three different learners. Nine new frames, five for 

- giving extra practice on one of the concept symbols 
(T), and four designed to provide a model oftheter- 
minal problem Solving performance, were inserted 
between the sixty-one introductory frames and the 


forty guided discovery framesto make a total of 110 
frames, 


Individual frames were dittoed on 5 x 9- inchcards 
with answers on the backs of the cards. Frameswere 
Scrambled by use of a table of random numbers, and 
a different scrambled sequence was developed for 
each of the sixteen students in that group. Reverse 
order sequences were formed by merely arranging 
the cards of a logical sequence from back to front. 


Procedure 


Approximately twenty-three experimental students 
in each of the two algebra classes worked at their 
desks in the presence of the three experimenters 
about 45 minutes a day for two consecutive days, Each 
student had the 110 cards stacked ina covered folder 
on his desk, He was allowed to work on only one 
card at a time and inserted each finished card into 
а second folder. Students were given red ink pens 
to write with and were told to draw a slash mark 
through any blank they could not answer before turn- 
ing the card over to see the correct answer. By fo- 
cusing the results of their activity on helping the ex- 
perimenters obtain data to revise the program (all 
connotations regarding student grades were removed), 
students readily ‘‘played the game,’’ and no student 
was observed to be “cheating” during the two days 
of instruction. 


A posttest only, control group design was em- 
ployed. Control students were assigned tothe library 


during instruction, but returned on the third day for 
testing. 


Measures 


Independent variables were the three sequence 
treatments and IQ. The dependent variables consis- 
ted of time to complete the program, errors made 
in the program, scores on a test of the concepts 
taught in the introductory part of the program, scores 
on a transfer test of the terminal problem solving 
task (finding formulae), and an attitude measure, 
The test on the concepts presented in the sixty-six 
introductory frames consisted of ten tasks taken di- 
rectly from the program ( For example, ‘‘If Tn = 9; 


n 


by = ”), but applied to a new number series. 
| ce 


The reliability coefficient for this test, calculated by 
Stanley's Split- Half Procedure (1:316), was . 82. 


The transfer test of problem solving consisted 
of five new number series for which the student was 
expected to state a general equation that would give 
the sum of the series at any term, n. Three of these 
series were used as test items by Gagné and Brown, 
and the other two were developed by the experimenters. 
Responses to each of the five series were scored on 
a two point basisfor a total of ten points. This test 
was found to be exceedingly difficult for the students, 


and no attempt was made to assess its reliability in 
the present study. 


RESULTS AND DISCUSSION 


Program Measures 


Table 1 presents means and standard deviations 
for time to complete the program and errors on the 
program for each Sequence condition. Analysis of 
variance indicated that program error rates differed 
significantly (, 01) in favor of logical sequencing, but 
erences did not. The signifi- 


the logical version, Apparent 
correct responses to Some of the items depended parr 
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TABLE 1 


TIME TO COMPLETE PROGRAM AND ERRORS 
ON PROGRAM FOR EACH SEQUENCE CONDI- 
TION (N = 16 FOR EACH GROUP) 


Measure 
Sequence Time(Minutes) Program Errors 


Condition M SD M SD Error Rate 
Logical 78.5 15.3 38.2 16.6 35 % 
Scrambled 80.9 13.6 54.8 17.0 50% 
Reverse 81.5 6.5 52.1 13.8 47 0 


knowledge of previous items along the lines of 
а knowledge hierarchy delineated by Gagné (3). Ran- 
domizing or reversing this logical sequence caused 
students to make significantly more program errors, 
This probably would not have happened if the items 
had been independent of each other. A list of dis- 
crete facts about the anatomy of the ear or a list of 
spelling words should generate the same error rate 
no matter how the items are sequenced. 


FIGURE 1 
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An examination of program errors broken down 
into blocks of ten frames for each of the sequence 
conditions reveals some interesting patterns. Fig- 
ure 1 contrasts program errors for the students un- 
der the logical condition. The erratic rise and fall 
of the logical line can be explained in terms of the 
program content. The first fifty frames present to 
the learner the introductory concepts (term number, 
term-value, sum) needed for the problem solving 
skill taught in the last forty frames. The mean num- 
ber of errors in each block of ten frames rises 
steadily as students begin to ‘‘drop out’’ in the con- 
ceptual hierarchy. Errors decrease forthe next 
twenty frames (51-70) because these frames review 
introductory concepts and provide a model of the ter- 
minal problem-solving performance. The last four 
blocks of ten frames (71-110) correspond to Gagné's 
guided discovery program in which he leads students 
to ‘‘discover’’ the formula to each of four different 
number series. The sharp decrease in errors on 
the second series (frames 81-90) is probably due to 
the fact that this particular series has a relatively 
simpleformula (n2). The means for the scrambled 
sequences, as would be expected, show very little 
variance and tend to decrease slowly as more and 
more information becomes available to students. 


Figure 2 contrasts program errors for the logi- 
cal sequence with those of the reverse sequence 


MEAN NUMBER OF ERRORS PER BLOCK OF TEN INSTRUCTIONAL FRAMES FOR THE LOGICAL AND T! 
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FIGURE 2 


MEAN NUMBER OF ERRORS PER BLOCK OF TEN INSTRUCTIONAL FRAMES FOR THE LOGICAL AND THE 
REVERSE SEQUENCE CONDITIONS (REVERSE SEQUENCE HAS BEEN TURNED AROUND TO CORRESPOND 
TO LOGICAL SEQUENCE. ARROWS INDICATE DIRECTION IN WHICH STUDENTS WORKED THROUGH PRO- 
GRAM. N=16 FOR EACH GROUP) 
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turned around. In other words, errors of the re- than controls on the Transfer Test, but not on the 
verse order students are shown from frames 1 Test of Introductory Concepts. None of the three 
through 110althoughthe students worked these frames sequence groups differed significantly on either test 
beginning with frame 110 and ending with frame 1, 
It will be noted how well the reverse order error rate, 
when turned around to correspond tothe logical order, 
matches the logical order *'dip-for-dip," but at a 


aed higher level eons The striking TABLE 2 

similarity between the two lines wasrather surpris- 

ing to the authors. One possible explanation is, of PERFORMANCE MEASURES ON TEST OF INTRO- 
course, that many of the frames in the program are DUCTORY CONCEPTS AND TRANSFER TEST OF 
over-cued. PROBLEM SOLVING SKILL FOR EACH SEQUENCE 


CONDITION AND CONTROL GROUP 
=———————————— 


Measure 


Post Program Measures 10 Item Test of 10 Point Trans- 
Introductory fer Test of Prob- 

Means and standard deviations of performances Concepts ISI ig Skil) 

relative to the two posttests for each sequence con- Sequence = Ss - 

dition are given in Table 2, For both tests, analy- Condition N M 5р М ED 

sis of ence indicated significant differences _——————————— 

among the means (р<. 01). Posteriori comparisons 

however, using the Scheffe method (2:296) to hold the” Logical 15 5.5 3.1 3.3 3.0 

probability of а Type I error constant, reveal that 

only the logical sequence group performed signifi- Scrambisd 1g 41 2.9 1.8 3.0 

cantly better than the control group on both tests, 

The scrambled Sequence group performed signif- Reverse 16 2.4 3.5 2.6 2.1 

icantly better than controls on the Test of Introduc- 

tory Concepts but not on the Transfer Test, The re- Control 16 08 10 0з от 


verse sequence group performed significantly better 
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fromeachother. The Kruskal-Wallis rank test also 
failed to reveal any statistical differences between 
the logical, random, and reverse sequence groups 
(2:362). 


The possibility of IQ and sequence interaction ef- 
fects on the Test of Introductory Concepts was ex- 
amined by means of afactorial analysis of variance. 
Non-significant effects for sequence and sequence 
x IQ interaction were indicated. The F ratio for IQ, 
however, was 4, 68 (p< .05). Figure 3 presents 
means of each sequence group on the Test of Intro- 
ductory Concepts after the groups had been split at 
the median IQ, The parallel lines indicate little or 
no interaction effect, and the higher means favoring 
the logical sequence group are not significant. It 
seems that aptitude, rather than sequence, deter- 
mined how wellan individual learner would do under 
all three sequence conditions. 


An attitude measure administered by the class- 
room teacher the day following the study revealed 
an interesting difference. Students inthe logical se- 
quence group tended to consider the program “‘inter- 
esting,” whereas students in the scrambled and re- 
verse treatment groups tended to feel ‘‘neutral’’ 
about the program (Chi-square = 6.6, p< .05). 
No differences were found relative to how difficult 


FIGURE 3 


the students felt the materials and the tests were. 
That the logical students felt the program was more 

interesting than scrambled and reverse order stu- 
dents would be expected because their error rate on 
the program was significantly less. The program 
was not ‘‘punishing’’ for students under the logical 
sequence condition. It is also possible the logical 
ordered students professed higher affect toward the 

program because they were able to follow the sim- 
ple-to-complex development from concepts and prin- 
ciples to problem solving skills, whereas students 

in theother two groups never were able to figure out 

what the program **was driving at" even though they 
learned to perform some of the tasks taught in the 

program. 


The data of this study reveal no significant differ- 
ences between logical, random, and reverse order 
sequence conditions relative to instructional time 
and posttest performances (although it should be re- 
membered that the logical group students were the 
only ones who performed significantly better than 
non-instructed controls on both posttests). The ex- 
pected interaction between sequence and IQ also 
failed to materialize. In this study there were no 
significant differences between sequence groups rel- 
ative to cognitive outcomes, The significant differ- 
ence in learner affect, however, may be enough to 
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justify carefulattention to Sequence in short instruc- 
tional programs. 


In reflecting upon the results of the investigation, 
the authors have come to the same conclusion Gagné 
has advocated in many of his studies: What is learned 
is more crucial in learning effectiveness than how 
it is learned (4:320). It must be remembered that 
all subjects responded to the same stimuli (frames) 
even though the frames were presented in various 
orders, The students in the Scrambledand reverse 
order sequence groups apparently were capable of 

integrating and organizing information regardless of 
the sequence of presentation, Insteadof studyingthe 
effect of frame sequence in short instructional pro- 
grams, perhaps more could be learned by studying 
the effects of deleting key frames related to certain 
tasks in a validated knowledge hierarchy and replac- 
ing these frames with placebo frames, (This was 


in “Тһе Acquisition of Knowledge” was derived by 
Gagné, and the present investigators were unable to 
find much correspondence between the program and 
the hierarchy. ) Though sequence effects would be 
expected to appear in fairly long instructional pro- 
grams, the present study, as wellas previous studies, 
lead the authors to believe that for Short (less than 
two hours) programs, sequence may not be as cru- 


cial to cognitive outcomes as has previously been 
thought. 


FOOTNOTES 


l. Paper presented at the California Educational 
Research Association Annual Meeting, Berkeley, 
California, March, 1968, 


2. The cooperation of the Montebello Unified School 


District during the study is greatly appreciated 
by the authors. 


3. The authors areindebted to Dr. Gagné for allow- 


ing them to use the learning program, Number 
Series, 
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ABSTRACT 


Aninstrument, the Schwartz Early Mathematics Inventory (SEMI), wasdevisedto assess, by means of group 
administration, the mathematical achievement of children ages 3 through 5. The data presented concern on ly 
the first stages of the process and deal only with 215 Kindergartners. A reliability of . 94 was obtained on the 
eighty-one items administered on 64 pages. The mean for items correct 43.2, The proportion of correct 
responses for various items ranged from .85 to.12. The children had satisfactorily acquired the physical 
and emotional attributes necessary for completing the inventory, namely, holding a pencilor crayon, turning pages, 
recording responses in designated areas, and maintaining satisfactory attention for the 12 to 28 minutes required 


for administering each of the three portions of the SEMI. 


THE DEVELOPMENT of procedures for theassess- 
ment of the capabilities and the measurement of ac- 
complishments of five-year-olds is a major challenge 
to the educational world. The Researchand Develop- 
ment Center in Educational Stimulation, University 
of Georgia, hasunified its total program of activities 
around early and continuous intellectual stimulation 
of children, ages 3 through 12. To carry on such a 
program requires a sound basis of evaluation for 
making decisions. 


The principal purpose of this research was to de- 
velop an instrument which could systematically as- 
sess the variations in performance, involving the ac- 
quired mathematical concepts and skills of kindergar- 
ten children. 


BACKGROUND FOR THE STUDY 


Neither child growth and development specialists, 
mathematicians, nor professional educators seemto 
be inaccordas to what mathematicalconcepts a child 
has developed at the preschool level or what the cur- 
riculum should encompass at thisage. Researchhas 
been limited in scope and the results do not lead one 
to make conclusive decisions, Opinions differ, some 
lead one to believe that the child has a very superfi- 
cial concept of numerosity while others, pertaining 
to children of a socially biased culture, state that an 
extensive background of mathematical acumen has 
been acquired prior to that developed by the organized 
school program. 


The studies of the developmental stages by Piaget 
(9) serve as abasis for muchdiscussion and argument 
concerning the mathematical abilities of children, 
including ages 4 and beyond. Dodwell (2 and 3 Ж 
Holmes (6), Inhelder (7), Elkind (4), and Wohlwill 


(12), have done replications of Piaget’s studies and 
have tended to support his findings, but in so doing 
these researchers have had to improve upon the in- 
vestigative procedures. Research by Estes (5), how- 
ever, refutes many of the findings mentioned. The 
author suggests a careful Scrutiny of the book 
Wallace (10) as a source of not only the studies men- 
tioned but many others which are relevant. Although 
most of these studies indicate some evidences of Stages 
of development, no agreement appears. 


Recent studies substantiate the statements that 
children come to kindergarten with considerable 
knowledge upon which the School can build an 
esting, challenging, and Sequentialcurricu 
Studies by Sister Josephina (8) and Brace andN 
(1) involved individual evaluations of Children, 
first studied thirty children inone kinde: 
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span, (b) assurance that the response of the individu- 
al can be accurately recorded, and (c) evidence that 
the response is individualratherthan a **cooperative'" 
endeavor. 


DEVELOPMENT OF THE INSTRUMENT 


In preparation for the developmentof the Schwartz 
Early Mathematics Inventory (SEMI)! three prim- 
ary sources were used to develop concepts and the 
initial item pool. First,a literature survey was made 
of material regarding individual or grouptesting 
which might have implications for measuring mathe- 
matical achievement. Tests, whether general or for 
Specific subject areas, were investigated regarding 
such areas as: format, manner of presentation, giv- 
ing directions, and recording pupil response. Avail- 
able materials and workbooks, methodology books, 
early childhood curricula, and general child growth 
and development releases werealso included. A pan- 
el of educators reviewed the items. 


Second, the author conducted individual assess- 
ments of 268 kindergartners, 


Third, kindergartenteachers were interviewed in 
respect to their number experiences with children. 


Since small muscle control must be taken into con- 
sideration, it was determined to use but one vertical 
line, ina majority of situations, for responding to each 
item. Whenever feasible sets were represented by 
circles and squares so that required vocabulary would 
be reduced for those with backgrounds of cultural de- 
privation. 


As a result, eighty-one items* were se lect- 
ed having a range from easy to difficult within speci- 
fic areas. Theareas rangedfrom simple to complex 
in mathematical learnings which might have probable 
relevance to this age group. Items were presented 
in numerous ways to give variety and to ascertain 
adaptability of the children. 


Pre-inventory exercises of approximately 15 min- 
utes duration were given to the children in order to 
familiarize them with the social setting, thetestcar- 
rels and mechanical preocedures. The children were 
askedto mark with pencilor crayon, whichever was 
most frequently used in the classroom, and were in- 
Structed to void any marks they did not want by mak- 
ingtwo horizontal marks across the vertical mark 
they had made, The double line was used to differen- 
tiate from X which many children were prone to use, 
In general, the test booklets contained one item per 
page-- eighty-one items on 64 pages--to avoidcom- 
plicating the task of responding by requiring even 
rudimentary reading skills. 


SUBJECTS 


Complete data were obtained from 215kindergart- 
ners residing in northern Georgia whose age range 
was 5 years 0 months to 6 years 5 months. The 
Children attended public and private schools from 
varied Socio-economic, racial, and ethnic back- 
grounds as follows: 


a. rural, low socio-economic children were ob- 
tained from the total kindergarten enrollment 


of one county in three schools, one of which 
was Negro. Total N=66. 


b. suburban children were obtained by the strati- 
fied sampling of the total enrollment of 5-year- 
olds in a county in a metropolitan area of ap- 
proximately one million population, proportion 
9 percent Negroes. Total N = 57. 


с. urban, low economic children were obtained 
from the total enrollment of three schools, se- 
lected under a Title I grant as being cultural- 
ly deprived, ina city of 18,000. Approximately 
50 percent of this group were Negro and were 
enrolled in one school. Total N - 60. 


d. urban, high socio-economic culturally advan- 
taged children were obtained from the total 
enrollment in a private, all white kindergar- 
ten. Total N -32, 


ADMINISTRATION OF THE SEMI 


The children were examined during the first 3 
weeks of February inroomsand hallways with which 
they were familiar, and in their own classrooms if 
their peers were assigned to carry on activities in 
another place. Thechildren were seated sixata table, 
separated by ‘‘carrels’’ of interlocking one-fourth 
inch plywood 1-foot high, witha minimum work Space 
of 19" x 32”. A maximum of twelve children were 
tested atany one period. Each group of six children 
were supervisedby either the administrator, a gradu- 
ate student or a teacher aide. The tests were adminis- 
tered by graduate students or the author but not by 
the classroom teacher, although she may have been 
an observerat an administration. The only assist- 
ance given during the time the inventory was being 
administered was to see that thechild was on the cor- 
rect page or to see that he understood where the re- 
sponse was to be placed, 


During the approximately 15 minute pre-inventory 
exercises, the administrator or the assistant gave 
any aid required or requested which would help the 
child in understanding the activities and the task to 
be accomplished. 


The items for the SEMI were read once to the 
children unless the task was Somewhat involved or 
the statement lengthy. The tempo of administering 
theitems was as rapid as feasible to eliminate 
**peeking?' talking and other distracting incidents. 
The average of two items per minute was suggested 
as a guide. In order to reduce fatigue, the SEMI 
was administered on Separate days, in three sittings 
besides the pre-inventory session, 


PRESENTATION OF THE DATA 


INVENTORY STATISTICS 


he general test data fi 


Ti $ 
ог the SEMI 
Table 1 as commited ү is presented in 


om data gathered from 2 15 
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kindergartners. The data indicates that the popula- 
tions did not fall into the normal curve pattern but 
are leptokurtic and skewed--a rather commonout- 
come with some kinds of empirical data. The kurto- 
sis is significant at the .05 level. The inventory re- 
liability equals . 94 using the Kuder-Richardson For- 
mula 20. 


The mean inventory score was 43.19. The scores 
earned by the children ranged from a high of 70 toa 
low of 8, The latter child omitted fifty-four items. 
Fifty-two children did not omit any items. The range 
of administration time for the SEMI was: partI, 12- 
25 minutes; part II, 10-18 minutes; and part III, 12- 
28 minutes. 


ITEM ANALYSIS 


An analysis of the eighty-one items of the SEMI 
is summarized in Table 2. The difficulty level of 
each item is indicated in terms of the percentage of 
correct responses. The proportion of the children 
selecting a correct response for particular items 
ranged from . 85 to.12. Seven items can be consid- 
ered comparatively easy as they were answered cor- 
rectly at least by 80 percent of the children. Eleven 
items canbe considered relatively hardas they were 
answered correctly by less than 30 percent of the 
children. No items fell within the extremely 
easy (p >. 95) and the extremely difficult (р <. 05) 
range. 


TABLE 1 


MOMENTS OF THE SEMI 


Mean: 43.19 + 1.02 
Standard deviation: 14. 97 
Skewness: -0.42 

Kurtosis: -0. 53 * 


* Significant at . 05 level 


Point-biserialcorrelations were computedfor each 
item and total test score. They ranged from . 62 to 
.11. Items with point-biserial coefficients above 
. 40 were considered to discriminate effectively. The 
coefficients for forty-nine items fell into this category. 


FACTOR ANALYSIS 


Although the SEMI developed as a Single- score 
general mathematics inventory, inter-item tetra- 
choric correlations were processed by factor analy- 
Sis with varimax rotation. These inter-item cor- 
relations varied from . 82 to random negative values 
as large as -,16, The number of factors to be ex- 
tracted was programmed at ten. The resultant ro- 
tated factor loadings ranged from . 80 to -.4lbut 
84 percent of the items had their highest loadings on 
the unrotated first factor and 52, 2 percent of the com- 
mon variance is accounted for by this factor, sothe data 
did not lend themselves to interpretation by multiple 
discrete factors, Factor loadings of. 40 or greater, 
on the rotated factors were given greatest emphasis 


in assigning the items to test parts and sub-parts. 
By combining these data with a rational analysis of 

the inventory, the items were categorized into four- 
teen comparatively recognizable and workable parts 
or sub-parts as given in Table 2. (See page 74). 


GENERAL OBSERVATIONS 


The 82" x T" format was adequate for the illus- 
trations and materials presented. Presentation of 
the items in mimeographed two dimensional form did 
not appear to be a barrier to understanding what was 
expected. 


TABLE 3 


ITEMS CONSIDERED COMPARATIVELY HARD-- 
PERCENT OF CHILDREN ANSWERING 
CORRECTLY < 30 


Item Percent 


24, Frame has more squares than 
circles, mark extra squares 24 


31. Mark the frame that shows 1/3 of a pie 24 


33. Mark the jar that is } full of water 19 
37. Mark each block/thing which has some 

black and some white 28 
38. Set of different shapes, mark those 

with more than two sides 14 
45. Seriation--sets of lines, make marks 

(3) for missing set 26 
46. Seriation--sets of lines, make marks 

(4) for missing set 21 
54. Bar graph, fillinor mark to look right 14 


58. Make 2 more таг 


ks than circles 
(2 circles) 


20 
80. When I count 5,4,3,2, mark num- 
ber I should say next 29 
81. When I count 10, 9, 8, 7, mark 
number I should Say next 29 


The response to most items was 
ate and the answer read: 


SUMMARY AND GENERALIZATIONS 


An instrument 
Inventory (SE МІ) 
of group administrati 


» the Schwartz Early Mathematics 


assess, b 


deal only with 215 five-year-olds 


"70 


items administered, 


.2. The proportion of correct responses for vari- 
ous items ranged from , 85 to .12 


SCHWARTZ 


A reliability оғ. 94 was obtained on the eighty-one 


the usual kindergarten tables, a very high degree of 
individual response was maintained. 


nt. Since the youngest children taking the SEMI 


Were able to perform the phy sical functions necessary, 
the author is encouraged to validate the instrument 


for 


three-and four-year-olds. 


FOOTNOTES 


1. 


The research reported herein was performed as 
а Special project as а portion of the postdoctoral 
education training program Sponsored as part of 
the activities of the Research and Development 
Center in Educational Stimulation at the Univer- 
sity of Georgia pursuant to a contract with the 

. 5. Department of Health, Education and Welfare 
grant No. OREG 2-6-061881-1406, 


A complete Copy of the Schwartz Early Mathemat- 
ics Inventory and related materials are on file at 
the Research and Development Center in Educa- 
tional Stimulation, University of Georgia. 


Grateful acknowledgment is made to Dr. Warren 

G. Findley and staff of the Research and Develop- 
ment Center in Educational Stimulation, for the 

assistance offered in conjunction with the postdoc- 
toral training program. 


Numeral number connotations are used interchange- 
ably because of usage by children, parents, and 
some teachers, 
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”4 
p TABLE 2 
GROUPING AND ANALYSIS OF ITEMS OF THE SEMI 
| Test Item Description of item--reflects instructions Percent * Point ** 
read to children correct Biserial 
Correlation 
| PART 1. PROBLEM SOLVING 
А, Number stories (If-then situations) Example: While 
demonstrating, ‘‘If I have two books and I take one a- 
way, then how many will I have left ?” 
Е 63. 2-1-n Objects pictured. Mark the right frame 81 51 
64. 5-2-n Objects pictured. Mark the frame 70 49 
65. 1+1=n No objects. Make right number of marks 60 3" 
61. 2+1=п Objects pictured. Mark the frame 60 41 
62. 3«3-n Objects pictured. Mark the frame 58 44 
67, 2-1-n No objects. Make right number of marks 55 35 
68. 5-1=п No objects. Make right number of marks 47 54 
” 66. 2+2=n No objects. Make right number of marks 47 51 
B. Reproduction. 
43. Make 2 marks 79 45 
44, Make 5 marks 68 45 
57. Make the same number of marks as circles (2) 61 51 
59. Make one less mark than circles (2) 52 21 
51. Make marks for combining a frame with a set of 
1 circle with a frame with a set of 2 circles 42 45 
58. Make 2 more marks than circles (2 circles) 20 3" 
[ Identical items on different pages ] 
60. Make 2 more marks than circles (2 circles) 20 33 
[ Identical items on different pages ] 
PART IL VISUAL DISCRIMINATION 
EAL DISCRIMINATION 
А, Matching and Analyzing Shapes 
19: Mark two numbers or figures that are the same 82 53 
y 14. Mark two numbers/figures that are the same 78 62 
28, Mark something that measures table/how tall you are 66 46 
21. Mark something that measures days of month 62 
42 
55. Mark the two shapes that look alike 
74 54 
56. Mark two figures that look alike turned the same 39 
B. Recognition of sub-sets = 
9. Mark frame that has 3 Squares 
81 
> 50 
41 Mark 3 of the squares in the frame 


48 


SCHWARTZ 


Description of item--reflects instructions 
read to children 


Percent * 
correct 


Point ** 
Biserial 
Correlation 


Two frames, if one more than other mark it. If same 
number of circles in both frames mark each frame 


Mark frame that has 5 squares 
Mark 4 of the circles in the frame 


Mark 3 set of 4 squares (scored two ways) 


Analyzing patterns or classification 


Frame of sticks or heavy lines, mark the stick that 
doesn’t fit 


Frame of circles, mark the circle that doesn’t fit 


Four frames, mark the two frames that have the same 
number (3 squares) 


Four frames, mark each of the two frames that look the 
same (four pictures of ducks ) 


Four frames, mark the two frames that have the same 
number (9 circles) 


Frame alternate circles-squares as beads, mark circle 
or square which will make look right 


Frame, square, two circles, square, etc. as beads, 
mark circle or square which added looks right 


PART III. COMPARISON 
M O MPARISON 


Semi-concrete non-equivalent sets 


Mark the largest dog 
Mark the shortest pencil 
Which frame has more buttons ? 


Draw a line from dot to dot (dots 7 inches apart) 
Which frame has the most pictures ? 

Abstract non-equivalent sets 

Mark frame with set greater than 3 


Frame has more squares than 
squares 


circles, mark extra 
Mark frame with set equal to 3 

Which frame has the least number of circles ? 
Mark frame with set less than 3 


Frame has more squares than circles, mark 
extra squares 


75 
75 
73 


51 


78 
74 


59 


54 


53 


51 


50 


85 
84 
82 
80 
17 


73 


55 
41 
36 


36 


24 


48 
47 
58 
37 


39 
44 


65 


60 


61 


31 


45 


37 
40 
44 
44 
45 


38 


29 
24 
40 
39 


26 
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д 


Тезї Нет Description of item--reflects instructions Percent * Point at 
read to children correct Biserial 
Correlation 
jee Ordinality 
iv. Point to left. Mark the first duck. 84 45 
18. Mark the last duck. 67 58 
19. Mark the third duck. 37 42 


PART IV. GEOMETRIC IDENTIFICATION 


39. Mark each circle in the frame * 54 49 
z 36. Set blocks or things, mark each which is all black 41 62 
40. Mark each triangle 35 36 
37. Mark each block/thing which has some black and 
some white 28 49 
38. Set different shapes, mark those with more than 
two sides 14 33 
PART V. SPATIAL JUDGMENT 
53. Pictures two pieces String, one tight, one loose. 
Mark the string which you think is longer. 49 20 
69. Two jars same size. Water put in other jar, how high? 46 38 
bga 70. Two jars, one b dia. other, mark height water other jar. 34 11 
È 45. Seriation-sets of lines, make marks missing set (3) 26 35 
46, Seriation-sets of lines, make marks missing set (4) 21 30 
54. Bar graph, fill in or mark to look right 14 29 
PART VI. TIME IDENTIFICATION 
26. Four frames clocks/watches, mark six o'clock 49 49 
25. Four frames clocks/watches, mark twelve o'clock 45 36 
29. Clock face, made two hands show 6 o'clock 35 42 
PART VII. NUMERAL-NUMBER 
Я © CONCEPTS 
The children confronted with numerals 0 through 10 for following items: 
j : T1. Mark the number for 3 73 41 
12. Mark number for 6 64 59 
14. УЕ ger that shows number fingers one hand 
64 
| 73. Mark number that shows how many eyes you have 63 | 
76. Mark the number that comes just before 5 52 53 
15. Mark the number which Shows you don't have any 47 ag 
т 


Mark the number that Comes just after 7 


45 
43 
18. Mark the number that comes two numbers before 4 41 


27 
CC 
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Test Item Description of item--reflects instructions Percent * Point ** 
read to children correct Biserial 

Correlation 
79. Mark the number that comes two numbers after 3 32 22 
80. When I count 5, 4, 3, 2, mark number I should say next 29 38 
81. When I count 10, 9, 8, 7, mark number I should say next 29 37 


PART УШ. MONEY IDENTIFICATION 


48. Mark the frame that shows a dime 71 24 
49. Mark the frame... which of 4 coins will buy the most 

(3 cents and nickle) 68 33 
50. Mark which will buy the most (2 frames) 56 14 
47, Mark the frame that shows a nickel 54 13 


PART ІХ. FRACTIONAL RECOGNITION 


32. Mark the jar that is > full of water 41 38 
30. Mark the frame that shows { or one-quarter pie 35 17 
3i. Mark the frame that shows 1/3 of a pie 24 14 
33. Mark the jar that is 4 full of water 12 01 


* Percent correct is computed including errors and omits 


**Point biserial correlation is correlation of item with total test score 
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AFTER A SIGNIFICANT F IN ANOVA? 
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ABSTRACT 


From time to time, investigators have proposed various solutions to the problem of identifying all con- 
trasts in classic ANOVA design. A few years ago, Sparks (2) reviewed and then compared four of the newer 


procedures in their application to a completely randomized 4-method x 7-class design. Specifically, 
nificant differences between pairs of means were dealt with in the methodological comparisons. 


only sig- 
In this paper, 


Tukey's **Compact"' (5) and the Link-Wallace (3:119-22) ANOVA shortcuts are also applied, using Sparks’ data, 


TUKEY'S TEST 


TUKEY'S ‘‘compact 2-sample test” (5) has been 
described as the slickest significance check yet de- 
vised; experience with it compels one to marvel at 
itsefficiency and simplicity (1). Only a few minutes 
need to be devoted to various analyses, in contrast 
with all other methods. Distribution-free in ration- 
ale, itis a k-sample test for location, 


Taking a pair of samples at a time, say a and b, 
the compact test requires only visual inspection to 
find which group has the highest value, scoreor 
measure, If Group a has it, and the lowest value is 
in Group b, then add the number of values in Group 
a exceeding all those in b, to the number of values 
in Group b which are below all the values in Group 


TABLE 1 


SPARKS 4- METHOD x 7-CLASS MEAN SCORES 


а. The critical counts are as follows: 


І. For a total count of 7, the 2-tail test equiva- 


lence is for 5 percent level significance. 


П. For a total count of 10, the 2-tail test equiva- 
lence is for 1 percent level significance. 


Ш. For a total count of 13, the 2-tail test equiva- 
lence is for the ‚1 percent level significance. 


Sparks Data (2) 


The design data used for analysis were mean 
Scores of twenty-eight classes randomly assigned to 
four groups; the four groups were in turn randoml 
assigned to the four treatment methods, Rearran ei 
in Table 1, the mean scores are given, enm 


CLASSES 
1 2 3 4 5 6 T X's Sums Ranges 
Method a. 14.59 23. 44 25. 43 18. 15 20.82 14. 06 14. 26 18. 68 130, 75 11.37 
Method b 20. 27 26.84 14. 71 22.34 19. 49 24. 92 20.20 21.25 148. 77 12.13 
Method c 27, 82 24, 92 28. 68 23.32 32.85 33. 90 23.42 27.84 194.91 10.58 
Method d 33.16 26. 93 30. 43 36. 43 31.04 29.76 33.88 32.52 227.63 10.11 


Sum of Ranges = 44,19 


Testing the overall hypothesis that the means ar 
the results in Table 2. 


е equal (ul = u2 7 u3 = u4), 


an analysis of variance yielded 
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TABLE 2 


ANOVA RESULTS FOR SPARKS DATA 


Variant Sum of Mean 

Source df Squares Square F 
Among the 3 830. 19 276. 13 15. 93 
Methods 

Withinthe 24 416. 76 17.37 

Methods 

Total 27 1246. 95 


ThetabledF value at ће. 05 levelis 3. 01 and the 
hypothesis was therefore rejected. The questionis 
then begged, ‘‘Which treatment means are signifi- 
cantly different?” 


It is well known that most authorities are agreed, 
subsequent individual t-tests by pairs of means should 
not be performed (4). Sparks (2) also provides an 
excellent summary of the pitfalls. 


Method Comparisons 


Sparks’ analyses (2:346-8), applying methods con- 
tributed by Duncan, Tukey, Scheffe, and Dunn respec- 
tively, obtained significant pairs marked with (*) as- 
terisks as given below: 


А. Duncan's Multiple С, Scheffé's S- Method 


Range Test 
dw Ь. 
d. c. b. 
a = Ж gus a + а ns 
ВЕТА b. ж ns. 
Ss ee e. ns. 
B. Tukey's (earlier) ^ D. Dunn's «C^» Pro- 
W-Procedure cedure 
d с. b. ü 6; bh. 
a, Ж CR nos, 9. * * ns 
be * oc b. * 
C. ns. C. ns, 


Tukey's newer ‘‘compact test ^ (5) produced the 
results below, similar to his earlier W procedure, 
Duncan's Multiple Range test, and to Dunn’s “с” 
technique; further, his ‘‘compact’’ method is seen 
to be more powerful than Scheffé's S method in this 
case. However, two advantages are evident. Tukey's 
"compact"? determines the level of Significance and 


involves much less labor in application than any of 
the other methods, 


On the other hand, using Duncan's test increases 
the likelihood of committing a Type I error and is 
regarded by many as too sensitive where 
the k-sample problem exceeds four, 


E. Tukey's ‘‘com- and Significance Levels 


pact” (5) of Pairs 

d. c. b. d; 6; bs 
9 * ^ nom a. .001 .05 n.s. 
b, ж * b. .001 .05 
C. n.s. €. n5. 


Demonstrating Tukey's newer test and its efficiency 
with one pair of treatments, а and d above, we find 
the highest score іп гоу й, the value being 37, 04 and 
the lowest in row a, mean value of 14.06. We count 
Seven values (all) in d higher than the top value in a. 
We also count all seven means in a lower than the 
bottom d groupvalue. The sum is fourteen, signifi- 
cant level being . 001 as seen in III “critical count 
rule for Tukey’s **compact test." Likewise, taking 
the pair b and c, the highest mean is 33. 90 in c; the 
lowestis 14.71inb treatment. Therefore, the ‘‘high- 
er than" count in с is four; the “lower than’’ in b 
equals five, The total value count is nine, significant 
at the . 05 level, under rule I above. 


Experience from scores of trials with Tukey's 
“‘compact’’ in many different designs supports cer- 
tain observations. First, samples should be six or 
more in size, but the n's need not be equal. Second, 
wheretiesexist, one can simply cancel out such val- 
ues, whether highs, lows, or inbetween. Third, one 
must comment on the use of significance tests. If it 
is true the primary aim of a test is to dampen the 
zeal of the researcher, he is asked to confess that he 
wants only confirmation of what he already believes. 
Tukey's “compact” relieves the investigator of choos- 
ing the significance level; reduction of bias toward 
either Type Тог Type Il errors is obtained. The criti- 
cal counts and levels are called asthey’re found, not 


as one would prefer them, to acceptor reject hypothe- 
ses, 


Link-Wallace ANOVA 
———n2*ace ANOVA 


For those investigators who prefer i 
parametric pro- 
cedures in the k-sample problem, perhaps the best 
bet is the Link and Wallace shortcut to analysis of 


It is still by far the quickest 
and most powerful Procedure for all A 
The method enables the еа, 


dealt with, 
the variation measure, Е 
are also included for bot! 
its. Inthe present case, 

single” observations, one per cell, 


From the tabled values calculated by Link and 
Wallace (3:146) for numbers of groups, and the sam- 
n e sizes in groups, onefirstobtains the critical fac- 
Or value needed, For four groups, of size seven 
each, the critica] factor = , 95 and multiplying that 
value by the sum of all four ranges, (. 95) (44. 19) - 


SMAWLEY 


TABLE 3 


LINK-WALLACE METHOD CONFIDENCE INTERVALS 


тї 


95% Confidence 95% Confidence 


Sum Critical Limits on Sums Limits on Sum 
Method Difference Sums 5% -level Difference Mean 
Comparisons (Rounded) Xp’s allowance Totals Differences 
a&b 18 2.6 +42 -24 to 60 -3.4to 8.6 
* айе 64 9.2 +42 22 to 106 3.2 to 15.2 
ж айа 97 13.8 +42 55 to 139 1.8 to 19.8 
* b&e 46 6.6 +42 4 to 88 .6 to 12.6 
* b&d 19 11.3 +42 37 to 121 5.3 to 17.3 
c&d 33 4.7 +42 -9 to 75 -1.3 to 10. 7 
Inthe Link-Wallce significance matrix, the pairs FOOTNOTE 
marked (*) are all P < ‚05 for these same data. 
F. Link-Wallace Results 1. Sparks’ article (2) includes both original 
references and detailed steps for the four 
d. e. b. methods developed by Duncan, Tukey in his 
a * * fma) earlier W-procedure, Scheffe, and Dunn; hence, 
Be, ow that information is omitted in this text and the 
c. (n.s.) following references. 
The last column intervals in Table 3, under F REFERENCES 
above, are obtained by dividing the 95 percent con- 
fidence limits by n=7 ‘‘rows’’ in this example, i.e., 1. Basler, р.р. and Smawley, R.B., “Tukey’s 
column labeled E/n-rows. Statistical consistency Compact vs Classic Test," The Journal of Ex- 
is observed for the treatment pair comparisons a&b, perimental Education, 36:86-88, Spring 1968. - 
andc &d, where the confidence intervals (*allow- Б 
ances”’) include zero, for a non-significance proof, 
2. i a ined der Notes on the Prob- 
‘ultiple Comparisons in a Com- 
SUMMARY pletely Randomized Design,” The Toumai ot 
= заза nal of 
Applications of six analytic methods proposed, Pxperimental Education, 31:343 349, Summer 
after a significant F is obtained inthe k-sampleran- д 
domized design, have been discussed. Results were 
compared and differences among the methods were 3. 
observed. Certain interesting similarities were ob- py 0А ne el Siak Non rametric 
ed. Publishers m- "mo nterstate Printers and 
Servi Publishers, Inc. Danville, Illinois, 1957, 171 pp. 
Summing up, then, we seethatthe various results 
obtained with Duncan's, Tukey's earlier “Ww,” 
Dunn’s *'c,?' Tukey's newer “compact,” and the 
Link-Wallace ANOVA shortcut all agree. Scheffé^z ^ Educational aa Кога, 8. Fs “Book Reviews," 
S was found to be less sensitive, but using it would 27:883 Wi and Psychological Measurement 
lower the Type I error risk. Finally, if pressed for 94, Winter 1967. 
time, and many will ask, “һо isn't?", the Tukey 
newer test for location and the Link-Wallace short- 
cut-ANOVA analytic procedure will serve the investi- 5. 


gator’s needs with respectable “© 


practical power,” 
as Tukey has expressed it. 


Tukey, J.W., 
Test to Duckwor: 
rics, 1:31-48, 


“A Quick Compaet Tw 

k 0-8 
th's Specifications,» н PA 
February 1959, met 
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ABSTRACT 


i i College (Oklahoma) in the 

ident teachers enrolled at Kansas State University and Northeastern State ; : 
EL were compared on (1) maintenance of philosophic orientation, (2) adherenceto teaching attitudes, 
@ ЗЕНОН to student-teaching, апа (4) performance on related criteria. The GNC Scale of Logical Consist- 
ency in Educational Ideas determined their philosophic orientation. The Minnesota Teacher Attitude Inventory 


determined their attitudes toward potential teaching situations. Comparisons were made before and after student 
teaching by institution, teaching level, and sex. GNC, MTAI, and ACT levels, methods and student-teaching 
grades, and grade point averages were tested for intercorrelation. 


K.S. U. students were more logically consistent and em 
fore and after teaching participation. 


pirically oriented in their ideas about education be- 
institutions. N. S.C. 


Teaching attitudes were best maintained by the elementary students in both 
students showed greater intercorrelation among all the criteria. Low logical consistency 
by N. 5.С. students may signify disagreement between theoretical and practical educational views, Maintenance 
of teaching attitudes by K. S. U. elementary students suggests variable effects of professional courses. Agree - 
ment among academic and educational criteria by N. S. C. Students suggests a greater unaminity of educational 
ends and means. 


A COMPARISON of the teacher 
grams at Kansas State University and Northeastern 
State College (Oklahoma) was undertaken in the 
spring of 1967. The research conducted was to de- 
termine what, if any, differences existed between 
and among the student teachers with respect to: (1) 
the logical consistency with which they held ideas 
about education prior to teaching participation, (2) 
the extent to which they held commonly-ascribed at- 
titudes toward teaching prior to teaching participa- 
tion, (3) the degree with which they had maintained 
their logical consistency and teaching attitudes upon 
completion of their relative periods of teaching par- 
ticipation, and (4) the levels they achieved on scho- 
lastic criteria pertinent to their respective college 
programs. 


education pro- 


It should be noted that prior research with stu- 
dent teachers at K. S. U. has previously been con- 
ducted with regard to logical consistency, teaching 
attitudes, and related scholastic criteria (5, 4, 6). 
The results then obtained strongly indicatedthat 


inter- institutional research in these areas should 
be conducted, 


PROCEDURE 
ate 

The student teaching program at ка ina 
University required satisfactory рехіори semester 
common professional requirement of tary educa - 
hours of both its secondary and elemen dit were giv- 
tion students. Of these, 6 hours of cre’ ipation pe- 
en for their 7-week actual teaching partie ry SUNT 
riod in the public schools during the КОДЕН? teach- 
The remainder was earned, prior to stu arrent with 
ing, in educational psychology, and concu: d either 
student teaching, in educational sociology an 2 1 le- 
(1) secondary principles and methods, or n 
mentary reading and methods. Supervision ae 
student teachers was conducted by staff members 
of the College of Education specializing in the par- 
ticular teaching fields represented. 


The student teaching program at Northeastern 


— M OY 
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TABLE 1 


COMPOSITION OF THE STUDENT TEACHER SAMPLE 


Kansas State University 


Northeastern State College 


OUS Number Total Percent* Number Total Percent* 
Grouped by Used Number Used Used Number Used 
School 90 142 60 194 298 65 
Secondary 49 77 64 119 187 63 
Level 
Elementary 41 65 63 15 112 67 
Male 15 26 58 76 121 63 
Sex 
Female 15 116 65 118 177 67 


*All student teachers on whom the necessary test, professional, and academic data was obtainable are 


included in the study. 


State College required completion of from 21 to 30 
hours in its professional program for both second- 
ary and elementary students. Of these, 12to 16 hours 
were granted to the secondary and elementary sen- 
iors enrolled ih the complete semester of full-time 
intern teaching. The remaining hours consisted of 
common courses in educational foundations, psychol- 
ogy, methods, and electives. Supervision of intern 
teaching was conducted by staff members of the Di- 
vision of Education on a geographical, rather than 
teaching field basis. 


For purposes of this study, the samples of ninety 
student teachers at Kansas State University and 194 
student teachers at Northeastern State College were 
dichotomized with respect to teaching levels and sex 
(Table 1). The logical consistency and philosophical 
orientation with which they held ideas about educa- 
tion was determined by use of the GNC Scale (Z). 
The GNC Scale utilized a force-sort technique to de- 
termine the extent to which the student teacher could 
separate fifty rationalistic statements (prior to the 
age of science) from fifty empirical statements 
(since the advent of science) relatin g education- 
al theory with practice. 


The teaching attitudes of the student teachers at 
both institutions were determined by the Minnesota 
Teacher Attitude Inventory (1). This scale was uti- 
lized for purposes of assessing each of the groups’ 
attitudes toward the future teaching situations in 
which they might find themselves. 


Comparisons were made between the two institu- 
tions on basis of GNC and MTAI scores attained 
prior to and upon returning from teaching participa- 
tion. This was done with respect to both the teach- 
ing level and the sex of the student teachers. Com- 
parisons on the same criteria were also made with- 
in the two schools in order to determine any signifi- 
cant changes inherent within them. 


An investigation was then conducted to determine 
the relationship of student teacher performance re- 
garding the ACT Battery, GNC Scale, MTAI, student 
teaching participation, cumulative grade point aver- 
ages, and courses in Principles of Education, Meth- 
ods of Teaching, and Educational Sociology. They 
were compared on the basis of their ACT perform- 
ance both within each school and between the two 
schools. Lastly, tests for intercorrelation a mon g 
performance on all criteria by the student teachers 


were conducted within each of their respective insti- 
tutions, 


PRESENTATION OF DATA 


The K.S. U. student teachers, prior to teaching 
participation, were logically consistent in their ideas 
about education (Table 2), With the exception of the 
male group, all GNC means were beyondthe , 01 lev- 
el, the elementary and female groups being the high- 
est. No N.S.C. student teacher groups were found 
to have attained a significant level of logical consist- 
ency (pS .05). The K.S. U. scores attained prior 
to student teaching were significantly greater than 
those of their respective N. S. C. counterparts ( Ta- 
ble 3). In both institutions the females were found 


to be more logically consistent than the males pri 
to student teaching (Table 4). EMO 


Although no differences in te. 
tudes were detected at K.S. U. 
ferences were found at N.S. 


ing level and sex (Table 4), 


Upon completion of thei 
ticipation periods, ay 


pective teaching par 
tency between the 5! 


only one shift in logi i 
gical con 
tudent teachers of the two sekadi: 


5 
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TABLE 2 


LOGICAL CONSISTENC Y OF STUDENT TEACHERS, AS MEASURED BY THE GNC SCALE 


Before Student Teaching 


After Student Teaching 


Student 
Teachers K.S. U. N.S.C. K. S. U. N. 2 
Grouped by mean mean mean mea 
School 4e 29 4g*** 30 
Secondary 45s 30 Age 30 
Level 
Elementary agree 28 46*** 29 
Male 38** 27 38** 29 
sa Female 49*** 31 4g*** 30 
*significant at . 05 level (GNC = 432) 
**significant at . 02 level (GNC = 437) 
***significant at .01 level (GNC = 441) 


was noted. The males no longer differedon this 

measure, although prior to student teaching the 

K.S. U. males had been higher (Table 3). All of the 

K.S.U. groups, however, were stillat the .02or 
higher GNC level, whereas each of the N.S.C. 
groups was still insignificant (Table 2). Inthe main- 

tenance of desirable teaching attitudes, the elemen- 
tary levels and females in both schools showed sig- 
nificantly higher scores both before and after student 
teaching (Table 4), 


With regard to intra-group comparisons, a con- 
sistent pattern was apparent. Prior to student teach- 
ing, the К. S. U, females had been significantly high- 
er than the males in logical consistency; the same 
situation held upon completion of this period. How- 
ever, a large increase in logically consistent males 
anda corresponding decrease in logically consistent 
female student teachers at N, S. C. was noted. 


A comparison of ACT Battery performance on be- 
half of the two schools also revealed highly signifi- 
cant differences to exist between the two schools 
(Table 5). In each instance the K.S. U. student 
teachers had achieved significantly higher scores 
on this test. The smallest of these was found in the 
comparison of the males in each school. Within 
each school itself, however, the only significantly 
higher scores on the ACT Battery were those of the 
N.S.C. Secondary student teachers as compared to 
their elementary counterparts. 


Several highly significant intercorrelationsamong 
the test, professional, and scholastic criteria with- 
in each school were detected. Among the N. S. ©. 
Student teachers, a total of forty pair of criteria 
correlated Significantly (Table 7); however, only 
twenty-six were found for the К. S. U. student teach- 
ers (Table 6). Among the K. S. U.students only Meth- 
ods correlated with Student Teaching, whereas it cor- 
related with all nine at N.S.C. The identical rela- 


by the student teachers. Th 
also existed in the same or 
tion, i.e., the secondar: 


tionship was found to exist with regard to the GPA 
criterion measure for each institution. Lastly, it 

was found that the first and second largest correla- 
tion coefficient in each school was, respectively, the 
score achieved among the MTAI and GNC tests taken 
before and after teaching participation. 


CONCLUSIONS 


A comparison of the spring 1967 student teachers 
at Kansas State University and Northeastern State 
College showed significant differences to exist in the 
logical consistency with which ideas about education 
were held by their respective student teachers. It 
had been earlier hypothesized,however, that exposure 
to rationalistic views dealing with the methods of ed- 
ucating man, on the other hand, results in conflict- 
ing frameworks of educational reference on the part 
of the teacher (4). 
evidencing signific 
of logically consistent Student 


theoretical and practical 5 
(3) a willingness to accept 


Significant differences w 
it 


i a ere detected in desirable 
udes displayed toward t 


heir future teaching tasks 
ese differences, however, 
der withineach institu- 
y level was lower than the 


the fe- 
males. Most of the elementary student teachers pi 
both schools were females, however (K.S. v 
cent n 
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COMPARISON OF LOGICAL CONSISTENCY AND TEACHING ATTITUDES BETWEEN SCHOOLS 


Before Student Teaching After Student Teaching 
Student 
Teachers GNC MTAI GNC MTAI 
Compared by t t t t 
K. S. U. 
School and 9. 10** te TES т. 98** 6. 33** 
N.S.C 
K.S. О. Secondary 
and 5.50** 5.82** 5.93** 4.56** 
N.S.C. Secondary 
Level 
K.S.U. Elementary 
and 7. 19** 5. 02** 5, 30** 4. 23** 
N.S.C. Elementary 
K.S. U. Males 
and 2. 26* 2. 85** 1.87 1.88 
N.S.C. Males 
Sex 
K.S. U. Females 
and 8.64** 6. 40** Tar 5.37% 
N 


.S.C. Females 


*significant at . 05 level 
**significant at . 01 level 


TABLE 4 


COMPARISON OF LOGICAL CONSISTENCY AND TEACHING ATTITUDES WITHIN SCHOOLS 


Before Student T hi i 
Student Teachers PESCE After Student Teaching 
Compared GNC MTAI GNI 
Within t à d миз 
K. S. U. Secondary 
and -1.47 =2./01* 0.16 - 
K.S.U. Elementary 2. 12% 
Level 
N.S. С. Secondary 
and 0. 88 -3.06** 
N.S.C. Elementary ` 0.21 -2, 01* 
K.S. U. Males 
and -2. 18** -3.00** 3 
K.S. U. Females 90 2. 81** -3. 14** 
Sex 
N.S.C. Males 
and -1. 67 -4.06** е 
N.S.C. Females 0. 43 -3. 13** 


*significant at . 05 level 
**significant at . 01 level 
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(2) more professional Courses, as required for ele- 
mentary preparation, are more effective in shaping 
teacher attitudes, or (3) the effectiveness of profes- 
sional courses for teachers may vary institutionally. 


With regard to differences in the educational en- 
vironments, the type and length of teaching partici- 
pation seemed to have little effect on the maintenance 
of either logical consistency in educationalthought or 
predetermined attitudes toward teaching. Consistent- 
ly smaller differences (smaller t- values) with re- 
spect to school, level, and sex onthese two measures 
seemed to indicate a possible reapproachment be- 
tween the two institutions. These diminishing differ- 
ences may thus be the result of (1) the longer, one- 
Semester student teaching period required of N. S. C. 
Students, as contrasted to the half- semester period 
at K.S.U., (2) different challenges to the prior- 
developed educational ideas and attitudes of the stu- 
dent teachers, (3) differences in the effect of long- 
term student teacher supervision done on either a 
level-basis (independent of geographicalrestrictions, 
as at K.S. U.) or a geographical basis (independent 
of level restrictions, as at N. S. C.). 


With respect to relationships between and among 
performances on pertinent scholastic criteria, the 
student teachers at the two institutions showed con- 
trasting profiles. Although the higher ACT scores 
of the К. S. U. students barely related to their higher 
GNC and MTAI achievement, the lower ACT scores 
of the N. S. C. students were, however, highly related 


TABLE 5 


to both of these tests. This may thus indicate that 

the ACT Battery of college achievement predictabili- 
ty neither measures logical consistency nor teaching 

attitudes very well. 


From an intra- school standpoint, the larger num- 
ber of significant correlations at N. S. C. isaccounted 
for primarily by more consistent relationships of 
Methods and Student Teaching grades with all other 
criteria. On the other hand, the total lack of rela- 
tionship in both schools between Student Teaching 
grades and GNC- MTAI scores may indicate some 
communality in the student teacher environments. 
This seems to be supported by the similar corre- 
lations among GNC- MTAI scores achieved by both 
groups prior to their teaching participation. It is, of 
course, also possible that the GNC and MTAI may 
not be significant predictors of success in the comple- 
tion of a teacher education program at either institu- 
tion. 


Lastly, more numerous and/or higher relation- 
Ships between general and professional scholastic 
criteria as noted at N. S. C. may be indicative of a 
greater campus- wide unanimity of efforts toward the 
specific goal of teacher-preparation. Consequently, 
it seems worthwhile to further investigate the rela- 
tionship which exists between the general and profes- 
sional educational elements of the multi-university , 
on the one hand, and the teacher-education oriented 
college, on the other. 


COMPARISON OF ACT BATTERY SCORES OF STUDENT TEACHERS BETWEEN AND WITHIN SCHOOLS 


Student Teachers Compared by 


Between Schools 


Within Schools 


t t 
School K. S. U. 
and 9, 15** — 
N.S.C. 
K. S. U. Secondary K.S. U. Secondary 
and 6. 43** and 1.57 
N. S.C. Secondary K.S. U. Elementary 
Level 
K.S. U. Elementary N. 5. С. Secondary T 
and 7.45** —— and 228 
N.S.C. Elementary N.S.C. Elementary 
K.S.U. Males K.S. U. Males 
and 2.73** and 30:88 
N.S.C. Males K.S.U. Females 
Sex 
K.S. U. Females N. S.C. Males 
and 9.27** апа 0. 98 
N. 5. С. Females N.S.C. Females 


*significant at . 05 level 
**significant at . 01 level 


t 
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TABLE 6 
INTERCORRELATIONS OF CRITERIA FOR STUDENT TEACHERS AT K.S. U. (N = 90) 


ACT GNC() GNC(2) MTAI(1) MTAI(2) Princ Meth Stu Tch Ed Soc GPA 
3 4 5 


1 2 6 7 8 9 10 
ACT 1 .232* . 274* . 182 . 029 .349** .385** -.040 .236*  .455** 
GNC(1) 2 ,697** .972** .305** .289** .012 -. 144 .217* .166 
GNC(2) 3 .338** .354** .304** .087 -.008 .308** ,287** 

„ MTAI(1) 4 . T25** .223*  -.089 -.043 227% . 119 
MTAI(2) 5 .133  -.215  -.015 .149  -.015 
Princ 6 .149  -.052 .271** .,872** 
Meth 7 .263* .218* .426** 
Stu Teh 8 .153 . 188 
Ed Soc 9 .295** 
GPA 10 


*significant at . 05 level 
**significant at . 01 level 


TABLE 7 


INTERCORRELATIONS OF CRITERIA FOR STUDENT TEACHERS AT N.S. C. (N-194) 


ACT GNC(1) GNC(2  MTAI()  MTAI(2 Princ Meth Stu T 
ch Ed 
1 2 3 4 5 6 7 8 ^ A 


10 
ACT 1 .460**  ,387** .197** .144* .297**  .240**  .199** .314**  ,419*« 
GNC(1) 2 .685** | ,429** .96T7** .326** .,270** .11 .310** | gage 
GNC(2) з .436** — .402** — .286** .181* 107 .258**  31g«« 
MTAI() 4 ."191** .232**  .347** 071 ‚ 164* T 
MTAI(2 5 .208**  .266** 114 ETT ES 
Princ 6 .334**  .9286g** .408** 562w 
Meth 7 245** .287** gage 
Stu Teh 8 .986%6 абвык 
Ed Soc 9 ТТ 
СРА 10 
| - *significant at . 05 level 


| **significant at . 01 level 
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BIOGRAPHICAL FACTORS AND 
TEACHER EFFECTIVENESS 


JOHN A. WILLIAMSON 


North Texas State University 


ABSTRACT 


This study attempted to detemine whether biographical factors might be identified which would distinguish 
between teachers receiving high and low student assessments. More than 3,000 high school seniors named the 
most and least effective teacher they had experienced. Most and least effective were defined in terms of the three 
major patterns of classroom teacher behavior identified by Ryans. Teachers named by three or more seniors 
Completed a biographical questionnaire. Effective teachers numberin, 

lresponded. A coefficient of correlation was found for each of the bi 
er rating. Significant correlations were found for amount of mother’ 
number of books read annually, home ownership, and student-teacher relationships. 


THE SUCCESS or failure of any school’s educa- to identify by name the one teacher who was the most 
tional endeavor rests largely with the classroom effective teacher they had encountered. They were 
teacher, and there is no substitute for an effective also asked to identify the least effective teacher they 
one, However, the identification of the effective had experienced. ‘‘Most effective’ and “least effec- 
teacher has been a problem that has plagued educa- tive” were defined in terms of the three major pat- 
tional leaders for years, andit has yet to be resolved, terns of teacher classroom behavior identified by 
Despite the voluminous research that has been devoted Ryans (4. The three dimensions are: 
to teacher effectiveness, results have been inconclu- 
sive and inconsistent. As Biddle (2) has pointed out, 1. friendly, under- versus aloof, egocentric, 
“Few, if any, facts are now deemed established standing behavior restricted behavior 
about teacher effectiveness, and many former ‘find- 
ings’ have been repudiated. ’’ However, few, if any, 2. responsible, busi- versus unplanned, slip- 
problems are more crucial in educationand, despite nesslike, systema- Shod behavior 
the paucity of definitive findings, efforts to under- tic behavior 
stand teacher effectiveness must continue. 

3. stimulating, imag- i 

The purpose of this study was to determine inative , nus aoe О ынчы 

whether biographicalfactors may be identified which astic behavior 


would distinguish between teachers receiving high 


and low student assessments. A total of 669 most effective teachers and 505 


А п least effective teachers w i 

Of major concern in any study of teacher effec- (** Most" and “Je effective will hereafter Se 
tiveness is the method of evaluation, Several ferred to as ене И and ineffective Respactiven 
techniques have been used in previous research, Of this number, 309 teachers ES named b y.) 
including pupil achievement, ratings by various or more seniors. Of these 309 teachers, 2 y ее 
observers, standardized instruments, and many located and were asked by Ша tones were 
others, However, because the student is the one graphical questionnaire, The questionnai pach са 
Person who observes the teacher in his day-to-day signed to gain some insight into the ba ire was de- 
teaching activities and, consequently, is in thebest the teacher and the opinions of the t ne 
Position to evaluate the effectiveness of that teacher, various aspects of teacher-pupil vanes regarding 
Student evaluation was used in this study, the 281 teachers, 125 of 149 Ee. aie v p 
PROCEDURES ineffective teachers responded. This represented a 

оне of 84 percent for the effective teachers апа 

A simple rating instrument was administered to 1 percent for the ineffective teachers, 

more than 3,000 Colorado high school seniors at In an attempt + ili 
о ut A 

their respective schools. The seniors were asked items which seemed Pede d Е biographical 
to consider every teacher whom they had encountered only twenty items in the questionnai o the Criterion, 
throughout their school experience and then inthis study, A Coefficient of co Н e included 


rrelation was found 


0 —————— 
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for each of the items and the criterion, teach- 
er rating. 


FINDINGS AND DISCUSSION 


Significant correlations were found for five of the 
items (see Table 1). The items were: amount of 


TABLE 1 


CORRELATIONS BETWEEN TEACHER RATING 
AND TWENTY VARIABLES 


Variable 2 
1. Teacher sex 0.01 
2. Number of children -0.04 
3. Teacher age 0.02 
4. Totalteaching experience 0. 13 
5. Mother's education 0: 18** 
6. Father's education 0. 01 
7. Number of siblings -0. 03 
8. Order of birth 0.11 
9. Size of home city 0. 05 
10. Decision to teach 0. 06 
ll. Distance home was from college -0. 03 
12, Father's occupation 0. 05 
13. Spouse's education 0.03 
14. Home ownership -0. 15* 
15. Send student to office for discipline 0. 24** 
16. Teachers do tasks willingly” -0. 04 
17. Pupils are competent to select dis- 
cussion topics -0.05 
18. Pupils try to do their best 0. 06 
19. Books read per year 0.22** 
20. Change of residence during last 
five years -0. 18** 
* Significant at the . 05 level. 
жж 


Significant at the . 01 level. 
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mother's formal education, status of home owner- 
ship, number of books read for pleasure each year, 
number of times residence had changed during last 

five years, and an opinion item regarding the teach- 
er’s attitude toward sending a pupil to the office for 
discipline. 


Results reported in Table 1 indicate that certain 
data from a biographical questionnaire correlate 
with teacher effectiveness. However, the correla- 
tions were rather low; consequently, fromthe stand- 
point of practical significance, they should be re- 
garded as only trends. The findings indicated that 
effective teachers were differentiated from ineffec- 
tive teachers in five ways: (1) effective teachers 
read more books for pleasure than did ineffective 4 
teachers, (2) the mothers of effective teachers had 
more formal education than did mothers of ineffec- 
tive teachers, (3) effective teachers changed their 
residence less often than did ineffective teachers, 
(4) effective teachers tended to own their homes 
more than ineffective teachers, and (5) effective 
teachers were inclined to blame themselves in the 
majority of cases when they sent a student to the 
office for discipline, whereas ineffective teachers 
placed most of the blame on the student. 


The apparent mobility of ineffective teachers as 
indicated by two of the findings is not surprising. 
That ineffective teachers would tend to move more 
frequently than effective teachers may be attributed 
to at least two factors: (1) The fact that students 
recognize the teacher's ineffectiveness should indi- 
cate that the school principal may also be aware of 
this and, consequently, this could result in the dis- 
missal of the teacher. Thisisarelevant factor des- 
pite the findings of Barr and Leeds that principals 
and students judge behavior of teachers differently 
(1, 3). (2) Ineffective teachers often experience 
poor teacher-pupil relationships which may create 
unpleasant and, sometimes, intolerable conditions 
for the teacher, Asaconsequence, in an attempt to 
escape this undesirable situation, the teacher may 
seek employment in another locale. 


That the effective teachers tended to blame them- 
selves in the majority of cases when they sent a stu^ 
dent to the office for discipline appears to indicate 
that the effective teachers are more aware of the 
feelings of the students than the ineffective teachers- 
It would also seem to be indicative of a more har mo~ 
nious relationship between the effective teachers and 
their students than the relationships existing between 
the ineffective teachers and their students. 


SUMMARY 


This study attempted to deter mine if relationships 
existed between student ratings of teachers and bio- 
graphical characteristics of teachers. Severalre- 
lationships were found. Mothers of effective teach" 
ers had more formal education than mothers of inef- 
fective teachers; effective teachers were less mobile 
than ineffective teachers, and effective teachers ех” 
perienced more harmonious relationships with stu- 
dents than did ineffective teachers. 


j^ 


WILLIA MSON 


APPENDIX A 


Questionnaire 


DIRECTIONS: Place the appropriate number in the blank provided to the left of the question. 


10. 


Sex 

1. Male 0. Female 
Number of children, if applicable: 
1. None 5. Four 

2, One 6. Five 

3. Two 7. Six 

4. Three 8. Seven or more 


My age is 

1. 20-24 5. 40-44 9. 60-64 

2. 25-29 6. 45-49 10. 65 or over 
3. 30-34 7. 50-54 

4. 35-39 8. 55-59 


Total number of years teaching experience 
excluding this year. 


My mother's formal education was that of: 
1. Eighth grade or less 

2. High school, but not a graduate 

3. High school graduate 

4. College, but not a graduate 

5. College graduate 


My father's formal education was that of: 
l. Eighth grade or less 

2. High school, but not a graduate 

3. High school graduate 

4. College, but not a graduate 

5. College graduate 


The number of brothers and sisters I have 
is: (Include living and deceased. ) 


1. Zero 4. Four-five 

2, One 5. Six or more 

3. Two-Three 

My order of birth in my family was: 
l. First 4. Fourth 

2. Second 5. Fifth or beyond 

3. Third 


During most of my childhood I lived in: 
1. The country 

2. A very small town (under 1,000) 
3. A small town (1,000-5,000) 

4. А town (5,000-10, 000) 

5. A small city (10,000-50, 000) 

6. A city (50, 000-100, 000) 

7. A large city (over 100,000) 


As you recall, was your decision to be a 

teacher made: 

l. Prior to or during high school 

2. After high school, but prior to college 
During college 

4. After college graduation 
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11, How far was your parental home from the 


institution which granted your Bachelor’s 
degree ? 

Within 20 miles 

. 21-100 miles 

101-350 miles 

. 351-1,000 miles 

. More than 1,000 miles 


л о он 


12. During most of your childhood and adolescent 


life, what was the general nature of your 
father's or guardian's occupation? 

l. Professional 

2. Farming 

3. Skilled (machinist, printer, etc.) 

4. Unskilled labor 

5. Business 


How does your spouse's educational level 
compare with you own? (Answer only if 
applicable.) 

1. My educational level is considerably 
lower. 

My educational level is somewhat lower 
My educational level is about the same 
My educational level is somewhat higher 
My educational level is considerably 
higher 


gu fm 


14. What is the status of your home ownership 


(home, duplex, apartment, etc.) ? 

l. Home is owned and fully paid for 

2. Home is owned and nearly paid for 

3. Home is owned and about halt paid for 

4. Home is owned and less than half paid for 
5. Home is being rented 


15. When a teacher sends a pupil to the office, 


what percentage of the cases do you feel are 
the fault of the teacher rather than the 

fault of the pupil? (Indicate опе.) 

- Less than 10 percent of the cases 

. 10-20 percent of the cases 

20-30 percent of the cases 

. 30-60 percent of the cases 


> бо о 


16. In light of your acquaintance with 


teachers, what do you think of the 
following statement ? 

'*Most teachers are willing to 
assume their share of the unpleasant 
tasks associated with teaching. ” 

1, Strongly disagree 

2. Disagree 

3. Uncertain 

4. Agree 

5. Strongly agree 
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TE. 


18. 


19. 


20. 
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What do you think of the following 
statement? 

**Pupils are usually quite competent 
to select their own topics for themes 
and speeches. '" 

l. Strongly disagree 


2. Disagree 

3. Uncertain 

4. Agree 

5. Strongly agree 


In light of your experience with 
children, what do you think of the 
following statement? 

** Most pupils try to do their work 
to the best of their ability." 

1. Strongly disagree 

2. Disagree 

3. Uncertain 

4. Agree 

5. Strongly agree 


How many books do you read for your 
own pleasure each year? 

1. Less than two books 

2. 2-5 books 

3. 6-10 books 

4. More than 10 books 


How many times has your residence 
changed during the last five years? 
(Do not include moves within the same 


city.) 

1. None 4. Three 7. Six or more 
2. One 5. Four 

3. Two 6. Five 


Note: 
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THE QUALITY OF WRITTEN EXPRESSION OF 


THIRD-GRADE CHILDREN UNDER 
DIFFERING TIME LIMITS 


MARY JO WOODFIN 


California State College at Long Beach 


ABSTRACT 


The purpose of this study was to determine whether significant diff 
pression of third-grade children would occur under differing time conditions. 
ren wrote compositions on topics of their own choosing once every 2 weeksforat 
One group wrote for 20 minutes; and one for any amount of time from zero to 
significantly higher than the other two groups in all five trials; the 90- 


20-minute group in all but Trial 2. 


TO ESTABLISH time priorities for teaching writ- 
ten expression, one of the questions educators may 
well ask themselves is what time allotments result 
in the highest quality of writing. Research evidence 
concerning the effects of time on the quality of writ- 
ten expression is scarce. One example is the work 
of Wilson (7) who found that third-grade children 
wrote longer compositions, on the average, if they 
wrote for 40-minute periods than if they wrote for 
25-minute periods, Noattempt was made to investi- 
gate the quality of thiswriting. Many suggestions are 
offered in the literature by individuals with extensive 
experience in teaching composition. Some recom- 
mend long periods of time (1, 2, 3, 4, 6); some 
short periods (8). 


To find whether the quality of written expression 
of third-grade children is significantly different un- 
der differing time conditions, a study of the quality 
of this writing under three time procedures was 
made: 1) Childreninthe first groupcould write for 
any amount of time they wished, from zero to 225 
minutes; 2) For the second condition, children 
Could choose any period of time they wished, from 
zero to a maximum of 90 minutes; and 3) The third 
time allotment was the customary classroom - one 
Where all children wrote for 20 minutes. Inall three 
ime Conditions, children were allowed to write on 


in topic they chose, and motivation time was not 
included , 


TI 
сашты ble was drawn from a large Southern 


а metropolitan school district. Children 


erences in the quality of the written ex- 
Three groups of third-grade child- 
otalof five experimental Sessions. 
225. The 225- minute group rated 
minute group significantly higher than the 


in the three groups did not differ significantly from 
each other in chronological age, IQ, socioeconomic 
rank, average language test Score, or initial quality 
of writing as measured by the McClellan Scale de- 
Scribed below, Initially, the 90 minute group scored 
significantly higher in reading than the 225 minute 
group;this difference was Significant at the . 05 level, 


Five trials were held 2weeks apart to see whether 
the relationship between time andquality was consis- 
tent over a period of weeks, Eachgroupwrote under 
the same time conditions throughout the study, 


A scale developed by McClellan (5) was used to 
judge the quality of the writing. This scale consists 
of nine rankings, with one the lowest and nine the 
highest, and rates imagination, expression of ideas 
and other creative elements. : 


Three judges rated each composition, 
age of their three scores was usedfor all si 
procedures, The aver 
judges was .79. 


The aver- 


tatistical 
абе correlation among these 


Average ratings on the McClellen Scale for the 
three time conditions are givenin Table 1, Analysis 
of variance and Duncan Test procedures showedthe 
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TABLE 1 


MEANS AND STANDARD DEVIATIONS OF RATINGS ON McCLELLAN SCALE FOR FIVE TRIALS 


225- Minute Group 


20- Minute Group 


90- Minute Group 


Standard 
Numberof Mean Standard Number of Mean Standard e о Кылы Deviation 
Trial Children Rating Deviation Children Rating Deviation Children a 
1 179 4.65 1.79 179 4.13 1. 61 184 3.54 1. 46 
2 184 5.19 1.74 181 4. 30 1.56 183 4.01 1. 48 
3 187 4.92 1.68 180 4594 — 141 186 3. 16 1, 40 
4 181 5. 20 1.71 187 4. 82 1.50 185 4,11 1.35 
5 174 5.45 1.52 179 4. 93 1.37 178 4. 08 1.30 
following significant differences amongthese means: REFERENCES 
l. The 225-minute group rated significantly l. Creed, Esther D., “аб ll We Write About?” 
higher than the 90-minute and the 20-minute Elementary English, 33:24-28, January 1956, 
groups in all five trials. 
2. Cross, Al „ ? i 
2. The 90-minute Sroup rated significantly higher Expression, "! pee S m Perg 
than the 20- minute group in Trials 1, 3, 4, January 1958 CREMA English, 35: d 
and 5. Alldifferences were significant at the i 
. 01 level except the difference in Trial 4 be- 
tween the 225- and the 90-minute groups 3. Diederi “ i 
whichwas sicat , * Diederich, Paul B, , ‘The Measurement of Skill 
gnificant at the , 05 level, in Writing, ” School Review 54:584-592, De- 
cember 19469, — — ——^ 
IMPLICATIONS 4. 
* Hardy, Hilda, бт Child, A Creator," Ele- 
mentary English 38:491-493, November 1961. 
If the primary aim of including written expressi 
as a part of the third-grade curriculum is ia Becr 85 McClellan, Jack, ‘Creative Writing Character- 
the highest quality in that writing, then the findings t pid Children, E unpublished EdD. scd 
from this Study would indicate that the best way to Sara Sity of Southern башны; е 
obtain this quality is to give children a time choice PE SUD 
within long periods of time rather thanto require all 
children to write for auniform short period, Quality 6. Stegall, C. m 1 One to 
of writing, as measured by the McClellan Scale Ten, „' Еа ак аана Jan- 
atleast, increases for third-graders with each in- чагу 1962. У English, 39: 
crease in time available to the children for writing. 
y^ 


The implications for individualizations of instruc- 
tion in written expression along choice patterns seem 
clear. And that allowing a choice of time for writing 
Over а period of time continues to result in better 
Writing, as measured by this scale, suggests further 
implications for classroom Scheduling. 


Wilson, Louis Ada, «A Study of Some Influencing 
Factors Upon andthe Nature of Young Children’ 5 
Written Language," The Journalof 


i Experimental 
Education, 31:4:371-380, Summer 


1963, 
Yocum, Charles G, “Creative Writi " 
Teacher, 81:52-53+, October Ming," Grade 
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ABSTRACT 


A method of determining the reliability coefficient of a test from a formulation which does not employ the 
Concepts of true score and error Score, together with assumptions about the process which generates variabili- 
ty in scores, is described. Several well-known reliability formulas as well as some new results are derived 
from models which hypothesize different sources of variability in scores. 


THE PURPOSE of this paper is to describe 
a method for constructing models based on assump- 
tions about the nature of variability intest scores, 
Novick (7) and Lord and Novick (6) characterized a 
strong model as one in which assumptions are made 
about the functional form of the distribution of errors, 
In the present paper the Concepts of true score and 


the expected value of X, then o2x - o2 (ЕХ + В)-о2в, 


concerning 02В аге the sa 
ing 02x, -In the present 
regarded as one in whichthe variability of observed 
Scores is specified in ter 
the number of items. 


Several different models are considered, The 
approach taken in this paper leads to а formulation 
which unifies some results given by other investiga- 
torsandindicates the relation among several specia] 
reliability coefficients, 


DEFINITIONS 


The subscript Е will refer to repeated measure- 
ments on a given person, and the Subscript j will 
refer to persons, Expectations, yariances, and co- 
variances will be denoted by E, 92, and COV, with 
subscripts indicating whether these values are taken 
over repeated measurements, persons, or both, 
The summation sign is not used. 


We consider a set of pairs of independent, re- 
peated measurements having the same distribution 
function, one member denoted by Хр} the other by 
хе. The quantity ХР} is a random variable having 
expectation over repeated measurements ЕХ р) апа 

g 
variance over repeated measurements ху. тһе 


expectation of Xi over both persons and repeated 


measurements is à Xgj, and the variance of Xyi 


over both persons and repeated measurements is 


— ИРА : и А 
Хе The distribution functions of Хе} and Xgj 


are the same for any person j, such that EXgj = 
5 


EX, and vx , = 02x! "n It is not assumed that 


8) 
g Е g 


these distributio, 


n functions are the same for diffe?" 
ent persons, 


Test reliability is defined as follows: 


Соу (х . Xt.) 
p. ti EU 
o. о X'. 
gj 8 gj & 
where 
pei ХЕ) =ЕЕХ X' -EE EEX' . 


А 20 я 
jg e EL oy Egg 
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Since repeated measurements are independent, 


EX,;X' - EX EX'. and 
g 9 ei Q8 si 


COV (Xgj» ХЫ) = E(EX EXij)- EEX :ЕЕХ'. = 


gi j & 9g jg El jg 5 
COV(EX,;, ЕХ) = e?Ex .. 
1 £^ р jg 


Also, 
2 2 2 
o2X,.= EEX2. - (EEX)? , 
gj © jg € jg 8 


2y . = EX2 2 
g^X,; = EX“, - (EX )^, 
А Б] g 8 g S 


Ex ee = 2 
Ec Хе) = EEX* - (ЕХ, ? , 


ig ig je 9 
and 
oP EX,; = Е(ЕХ, )2 - (EEX )? , 
ig jig jg 8 
Therefore, 


а) X= Bex + o?EX 


ei © jg Bj g 8 
Substituting o°EX,, for COV(X, ., X' ) in the defini- 
ШЫ Е gj fOr 29) ey g) int e defini 
tion of p, using the relationship с X.. = o Xt, 
si 9 gj 8 
and using equation (1) gives 
c?EX y sex : 
j g) 
ОР os $8 "5 wc Я 
oy o? 
. 8) gj 
85) gi 


Construction of a strong model, as considered in 
this paper, consists of the following steps. A pro- 
cess which generates variability in scoresis hypothe- 
sized, and the distribution function of the observed 
Score random variable, хь» is determined. That 


function contains parameters such as the number of 
test items or the probability of chance success. The 


value of the quantity Ec?X.; is then found, and, by 
Jg 


Substituting in (1) and (2), equations for the total 
Observed variance and the reliability coefficient are 
Obtained, A reliability coefficient found by this 
method is specific to the particular source of varia- 
bility in scores which is hypothesized. 


ltem Sampling Model: A Coefficient of Equivalence 


We consider first the model presented by Lord 
» In which variability in scores is attributed to 


(5) 


sampling of items froma population. Each measure- 
ment consists of drawing a random sample of N items 
from a population of items, a certain proportion of 
which, Pj, are known by person j, and the random 
variable кь] represents the number of items in the 


sample known by person j, Over repeated measure- 


ments хь; is distributed binomially with ox = 
S 


NP. (1 - Р.) and EX = NP.. f 
jc л) ni : zj j Therefore, 


ERE) 
{олу ©З а € ык. 
gj 
g N 
The arithmetic mean of Q1 over persons is 


NEEXgj - E(EX,;) 2 


TE ix 
i g) . 
jg N 
i 2 e о: 2 
Since с“ЕХ,; = E(EX,..) (EEX,..) 2, 
jg jg See 
2 
E X,,(N- Е X) c^EX,. 
A Я Y om 
ЕО?Х з= gj B sj “Ais 
Jg 


N N 


Substituting this result in (1) gives 


x 2 = 
(N De EX.. EX, (N EX, 
2 jeg gj gj 
(Q2) c Хр + 
gj N N 


Substituting the same result for Ео?х, in (2), not- 


i 
ing that s 
2 P 
ER 
= __, and solving for p, gives 
мох N 
gj 8 
E Xj(N- EX. 
дь be ra R 
Q Sj gj 
Bed 5: 
PPa 


which is the same as the Kuder-Ric hardson formul; 
21. Here, the subscriptQ indicates the value ASSUMER 
by the reliability coefficient when the only source of 
variability in scores is the sampling process. 


Substituting (Q2) in (2) leads to the following 
result: 
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2 
СЕХА 
Q4 pg - Ж 
S NE EXQ(N-EXj 
EX, j+ gj [2 
jg 
N-1 


Chance Success Model: A Coefficient of Precision 


Another model attributes variability in scores to 
chance success on multiple-choice tests. The selec- 
tion of items is not regarded as a source of varia- 
bility, and the results, therefore, are specific toa 
given test form, Let Cj be the number of items on 


a multiple-choice test known by person j, and let p 
be the probability of chance Success on each of the 
N- Ci items not known. Over repeated measure- 


ments with that particular set of items Xoj is bino- 


mially distributed with EXgj = Cj * p(N “Б апа 
g 

оҳ ‚ = р (1-р) (N- Cy. Solving the first equation 

g 8) 

for © and substituting the result in the second gives 


2 
(81) о. = p(N- ЕХ). 
e 8 g gi 


Finding the arithmetic mean of (S1) over persons 
and substituting in (1), we obtain 


(S2) ex -p(N- EX.) + c?Ex.. 


gj £i jg 9 
Substituting the arithmetic mean of (S1) in (2) yields 
pN- E Xgj) 
(83) 06=1- £j 
2. 
OX: 
ei © 


Here, the subscript .S indicates the value of the 
reliability coefficient when the only Source of 
variability in scores is chance success, 


Also, substituting (S2) in (2), we obtain 


ig 
(S4) Paik жес . 
o2EX,, + p(N- ЕХ) 
‚ар Kej 
ЈЕ £l 


From substitutions it also follows that 


(55) К: {= @-р)®%2с, + P(1 - р) (М - ЕС.) 
3 gJ j 1 j ye 


THE JOURNAL OF EXPERIMENTAL EDUCATION 


hi 
that oc; 
= ј 
(66) p. = , 
oc; + p(1 - р) (№ - ЕС) 
1 1 
and that 
(1 - p)?0?e, 
G7) pg = j`, 
2. 
° Хш 
£i) 


Equations (55) and (S6) were given by Carroll (2) and 
Horst (4. Equation (S7) indicates that, while Pg is 


2 


O"EX . 
j gj i 
i a 
equal to the variance ratio itis not equ 
o2x 
gj 8 
oc. 
1 
to the variance ratio And, while the cor- 
2 
gi 
85) 


relation between ЕХ and X , - EX ; over all per- 
g E 8 g 8 


Sons is zero, the correlation between ©, апа Ху-С, 


poC; 
j 
over all persons is – 2 For other equa- 
2 
9 (Xgj-Cj) 
£l 


tions related to ($5), (S6) and (S7) see Carroll (2), 
Horst (4), and Burkheimer, Zimmerman and 
Williams (1). 


Combined Model 

Consider, now, the distribution of observed SE tb 
Over repeated measurements when variability die 
from both of the above processes, Assume thatt Teo 
is, first, Sampling of N itemsfroma Doe аве 
sulting in an Xii value and, second, chance Suc 


own. 
on some of the МХ. items in the sample not kn 


Let the Subscript h refer to repeated p 
where the Second kind of variability 18 кее b th 
Hen Ха; is the observed score resulting from bo 


Sampling and chance success. 


Over repeated measurements the values of Хе] 


will be distr 


ibuted binomially as a result of sampling- 
For each po 


Ssible value of Xoj the values of Xni 
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Will also be distributed binomially, as a result of 
chance success, such that ЕХ = X_. + p(N-X 3) 
n ghi si gj 


2 
ando“X =p (1-р) (N - X ). The expectation of 
h Shj а) Р 


the entire distribution of Хер; values for person j is 


+ pN- EXgj), and the variance of 
g 


that distribution is 


‚ = Eo?x |, + 02ЕХ 


2, 
оёх 
su gh SU gh 


gh 


ghj 


=Ep(1 - p) (N-X,) + c? |х, + p(N - Xg) 
i a) ta pu gi 


Since o? |X,.. -x)| =02 | a -DXgj*N 
nce o К vj] А (1 -p)Xgj + Хр 


[4 
= (1 -p 202х j Я 
g 
2 2.2 
оёх = p(l- p) (N- ЕХ) + (1 - D^o^Xgj 
gh Eh g 8 g 8 


- 2 " - " 
(1-p)“EXg; (N-EX,;) 
g g 


N 


ра-р (N-EXpj) + 


(1-р) di [» * oci ] 


Substituting this ex- 


1-р 


g 
о2х gives 
gh 
E Xgnj(N- Е Хав) 
RI) охх " 8, , , 
gn? N 


Which has the same form as (Q1). Finding the ex- 
pectation of (R1) over persons and substituting in (1) 


also gives 
W- 002 E хац 
j gh 


c?x. hj = 


g,h,j x 


(R2) " 


E X4(N- E X) 
. “ghi . “ghi 
g,hj E g,hj b 


N 


Substituting in (2), we obtain 


p 


X N-E 
E at Хш, 


N 

(R3 Pp- —— |1-ghi 89 ghj 
N-1 2 

N c Xohi 

ghj 599 

and 

с? ЕХ 

jgh 

E X ,.(N- E Xni 

ei s E, ghj) 


ghj 


(R4) кый 
R4) p, = 
R уст? E Xgnj * gj yj 
j gh 
N-1 


Here, the subscript R indicates the value of the 
reliability coefficient when variability in scores re- 
sults from both sampling and chance success. Al- 
though equations (R1), (R2), (R3) and (R4) have the 
same form as (Q1), (Q2), (Q3) and (Q4), it should 
be noted that for given N and given values of Р, the 


quantity Е X. does not have the same value as 
; ghj 
ghj 
ЕХ, since E Xp; = EX, +p(N- E X,,),and 
, . Eh ^ Ш 
gi ” ghi 59 gy 9 gj © 


2 
that c Xghj is not the same as c?x. 


g, hj gj 


From substitutions the following e i 
obtained: g equations are 


(N-0-pc^x., 
2 jg 
(R5) о Xghj = è 
g,h,j N 


(1-p)(N- ЕХ) | Np + (1-р) ЕХ 1 


gj gj 9 
N 
с EX. 
Чеш 
R6)p. ig 
c 
EXgjlN- EX.) 
свх +в) oi | 
ee en: 
16 N-1 
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FEE no 
1=р 
(1-р) 70? EX, 
(R7) Pp = : ЈЕ : 
g,h,j Eni 


Therefore, while Dg as a function of ЕХ апа 
g,hjj 


о? Хо is givenby {һе same equationas Po ( the 
ghj 


Kuder-Richardson formula 21), PR as a function of 


c?EX . and o2x is given by the same equation 
> Е; ; ghj 
ig ghj 


as Ps. 


DISCUSSION 


The method of constructing models outlined in this 
paper indicates how it is possible to define reliabili- 


pointed out by Guttman (3). 
tribution functionof х hasbe 


Such as (Q3), (R3) and (s3 


) are obtained, which con- 
tain quantities Which can 


be estimated froma single 


be only approximated > 


Except for Lord's (5) derivation, the KR 21 has 
been obtained by first deriving the KR 20 and then 
assuming that all test items have the same difficulty, 
This assumption is implicit in Lord's model and in 
the procedure used in this paper, in that all items 
are regardedas being sampled from а common popu- 
lation of items. It follows that over repeated sam- 
ples the expected value of an item Score for person 
dis Pj Over all persons and all samples the ex- 
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pected value of an item score is EP; and that quan- 


tity is the same for all items. 


FOOTNOTE 
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the National Research Council of Canada puri 
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ABSTRACT 


Effects of proportion of Latin suffixes and of Content Area upon newspaper article comprehension of eighth 
Бгайегѕ, as measured by cloze procedure, were investigated using a 3x3 factorial design. Statistically reliable 
mean differences were obtained between cloze test performance on Science and Т. У. -- Theatre passages and be- 
tween such performance at High versus Medium and Low levels of Latin suffix density. The difference in mean 
Performance across Content Areas is explained in terms of other linguistic factors--particularly differential 


density of prepositions. 


READING COMPREHENSION may be viewed as 
а function of the reader's language competence, the 
Subject matter of the language message, and the syn- 
tactic and morphologicalcomplexity of the language 
message itself. It has been generally recognized (6) 
that comprehension is differentially affected by ma- 
terial in different Content Areas. In terms of the 
above conception of comprehension this effect may 
be a function of the necessary background infor mation 
required of the S, the author's writing style, the vo- 
cabulary specific to certain Content Areas, the inter- 
ests and attitudes of the reader or listener concern- 
ingthe various subject mattersof Content Areas, and 
the linguistic and morphological complexity with which 
passages in different Content Areas are generally 


generated (3, 5, 2). 


The purpose of this study was to determine whether 
passages concerned with different subject matter 
(Content Areas) differed in some dimensions of lin- 
guistic complexity and if so whether controlling these 
variables would result in differences in comprehen- 
Sion as measured by cloze test performance. The 
Study also investigated the relationship of Ss’ general 
reading ability andtheir comprehension of different 
Content Areas. 


PRELIMINARY ANALYSIS 


Five randomly chosen newspaper articles in each 
of ten different Content Areas from the Los Angeles 
“imes were analyzed, using three syntactic and mor- 
$ 9logical variables (dependent clauses, nominal 
n pdifiers, and the proportion of suffixes of Latin 
NI jin). These variables were chosen because they 
M t; Previously (1) been found to be the best predic- 
LN Sof cloze performance of fifty-nine suchvariables 


investigated. The mean frequency of occurance for 
each of the three variables for each Content Area is 
presented in Table 1. To test the reliability of the 


TABLE 1 


MEAN FREQUENCY OF OCCURRENCE OF THRE 
E 

LINGUISTIC VARIABLES IN PASSAGE: 

CONTENT AREAS PNE 


Content Latin Dependent Nomi 

п lominal 
Area Suffixes Clauses Modifiers 
Sports 15.0 27.2 2.2 
Human й 

Interest 15.6 31.0 5.0 
Fashion 18.5 32.0 3.8 
Arts 21.6 22.4 3.4 
T. V. -- Movies-- 

Theatre 22.8 30.4 4.0 
Religion 26.3 25.2 3.6 
Government 27.2 34.4 5.6 
Labor and 

Business 21.6 43.6 2.8 
War 27.8 33.8 4.6 
Science 30. 4 33.0 4.8 


2 


differences in means, Duncan Multiple Range Tests 
were applied. Theresultsof this analysis indicated 

that the proportion of suffixes of Latin origin (L- 
suffix density) yielded fifteen reliable differences 
between Content Area pairs while dependent clauses 
and nominal modifiers yielded five and ten reliable 
differences between Content Area pairs, respectively. 
Since L-suífix density was the most successful 
variable in discriminating Content Areas this vari- 
able was chosen as the independent variable for the 

main study. Three Content Areas were chosen for 
the main study on the basis that the mean of each was 
significantly differentfromtheothertwo. A second 

criterion for Content Area selection was that a wide 
sampling of passages with varying degrees of L- suffix 
density could be obtained to permit a factorial study 
of L-suffix density effects. The three Content Areas 
selected were Science, Human Interest, and T. V. -- 
Movies-- Theatre. 


METHOD 


Materials 


One hundred newspaper articles from each of the 
three Content Areas were taken from five daily news- 
papers (Los Angeles Times, New York Times, Chi- 
cago Tribune, Philadelphia Inquirer, and the Wash- 
ington Post). These newspapers were chosen because 
they represented a wide cross section of the largest 
dailies and were felt to be representative of news- 
paper writing. Twenty articles were chosen from 
each newspaper in each Content Area. All articles 
were chosenfrom issues of November and December, 
1967. Only the first 200 words (or the end of the 
sentence closest to 200 words) were considered for 
this study. The number of L-suffixes were counted 
and the proportion of L-suffixes per 200 words was 
calculated for each article, The criterion for deter- 
mining a L-suffix was whether it was so designated 
by An Etymological Dictionary of the English Lan- 
guage (9). Proportions ranged from .000 to . 200, 
Frequency distributions for each Content Area indi- 
cated that proportions tended to cluster at the high 
endfor Science and atthe low endfor T. V.--Movies-- 
Theatre with Human Interest in the middle, 


On the 
basis of the distributions, proportions of . 01 to . 05 
wereclassifiedas Low," ‚06 to , 10 as''Medium, ” 
and.11 to .15 as “High, ” 


Ten articles from each 
were chosen at random 
articles would be chosen 
Spapers for each Content 
possible to meet this cri- 
exceptions occurred, In 


Content Area at each level 
with the restrictionthat two 
from each of the five new 
Area and level. It was not 
terion exactly but onlyafew 


TABLE 2 


MEAN SRAT SCORES AT EACH CONTENT AREA AT EACH LEVEL OF LATIN- 
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Some cases it was not possible to obtain two articles 
at either the ‘‘ High" or **Low"' level in a content area 
from a given newspaper. 


PROCEDURE 


The articles chosen for the final study were made 
into cloze tests by deleting every fifth word and ee 
placing it with ablank of standard length. } Each ar z 
cle was tested on five Ss. Each S receiving a pa 
sage was given a clozetest wherein deletions begon 
with either the first, second, third, fourth, or fifth 
word. Therefore, every word of each article was 
tested once. Each S read and responded to two ERE 
cles during a l-hour testing period. Articles were 
randomly paired with the restriction that they pear 
be in the same Content Area and at the same d Д 
of L-suffix density. Ss were tested in groups ed 
proximately 25, E readthe instructions to the eni 
and negotiated a sample test with the Ss. All 55 the 
tested by one E and were given 40 minutes to comp E 
the two passages. Stanford Reading Achievement" 
Test scores on all Ss were obtained from school re 
cords. These tests were administered approximately 
6 months prior to this study. Twenty-five Ss were 


tested under each of the nine treatment conditions 
investigated. 


SUBJECTS 


The Ss were 225 eighthgraders from one Southern 
California school district. The district is a predomi- 
nantly lower-middle to upper-middle class commun“ 
ity. The Stanford Reading Achievement Test (SRAT) 
scores indicated that the sample included a wide range 
of reading ability. 

RESULTS 


Cloze items were scored correct only if an inser 
tion was the exact word deleted; accurate spelling was 
not required. The total score for each S was calcu 
lated by combining the two passages negotiated by 
each 5 and calculating the ratio or correct insertions 
over the total number of deletions. 


To ensure that the random assignment of Ss sd 
Content Area and levels of L-suffix density did TE 
result in group differences in reading ability, F 
SRAT scores were compared. A 3x3 analysis А 
variance showed no reliable differences bees 
groups. The means are presented in Table 2. acne 
means and the analysis of variance give strong ind! 


able 
tion that the experimental groups were compara 
on SRAT scores, 


Latin Suffix Dens ity 


SUFFIX DENSITY 


Science Human E PARE T E —— MER 
High 8.20 8. 34 8. 58 ни 
Medium 9.10 8. 07 7.80 us 
Low 8.89 8. 20 8. 26 $ 
Mean 8. 73 8. 20 8. 28 


AQUINO-MOSBERG-SHARRON 3 


TABLE 3 


MEAN CLOZE PERFORMANCE AS A FUNCTION OF CONTENT AREA AND LEVEL OF LATIN-SUFFIX 


DENSITY (MEAN PROPORTION CORRECT) 


Level of Latin- 


Suffix Density Content Area 
Science Human Interest Tq. -Movies--Theatre Mean 
High . 290 .292 .280 . 287 
Medium .346 . 292 337 
Low . 398 .302 .335 
Mean .345 .292 


Mean cloze test performance data are presented 
in Table 3, The mean proportion of correct responses 
is highestfor Science and lowest for T.V. --Movies-- 
Theatre. Mean performance was the same for Low 
and Medium levels of L-suífixdensity and somewhat 
lower for the High level. A3x3 analysis of variance 
(Table 4) indicated significant differences for both 
main effects (Content Area, L-suffix density). Spe- 
cific comparison of means using the Duncan Multiple 
Range Test showed that the only reliable difference 
between Content Areas was between Science and T.y-- 
Movies--Theatre. For L-suffix density, the High 
mean was significantly lower than either the Low or 
Medium means. 


TABLE 4 


ANALYSIS OF VARIANCE OF CLOZE SCORES 


Source df SS MS F 
Content Area(CA) 2 . 107 .054 3. 176* 
Latin-Suffix 

Density (LS) 2 .118 ‚059 3.471* 
CA x LS 4 .131 .033 1.941 
Error 216 2.509 . 017 
*p«.05 


The correlations between SRAT and cloze scores 
are presented in Table 5. Separate correlations were 
computed for each Content Area collapsed over L- 
Suffix density levels and for each level of L-suffix 
density collapsed over Content Area. Comparable 
correlations (,612and. 644) were obtained for Human 
Interest and T. V. -- Movies- - Theatre but a somewhat 
Mower correlation between Science and SRAT (. 515). 


i Similarly, comparable correlations were obtained 


\ bietween SRAT and Medium and Low L-sulfix density 


NY lelvels (. 651and . 610) but a somewhat lower correla- 


IN 
k 
х 


T between High and SRAT (. 508). 


To shed further light on the difference in cloze 
performance between Science and T. V. --Movies-- 
Theatre, several ad hoc analyses of the passages 
were performed. An analysis of the mean number 
of sentences per passage and the mean number of 
words per sentence revealed that Science passages 
contained fewer but longer sentences than T. V. -- 
Movies--Theatre. The mean number oí sentences 
per passage was 8. 86 and 10. 90for Science and T.V.-- 
Movies-- Theatre, respectively. The mean number 
of words per sentence was 22. 98 and 18. 86 for the 
two Content Areas respectively. Analysis of vari- 
ance indicated that both of these differences were 
statistically significant at the . 05 level[ F (2,87)- 
5.39], [F(2,87)-3. 42]. Certain modifiers such as 
prepositional phrases and relative clauses make sen- 
tences longer. It was hypothesized that since Science 
passages had longer but fewer sentences they would 
contain more prepositions. The mean number of pre- 
positions in Science passages was 29.0 and 23.7 
for T. V. --Movies--Theatre. Analysis of variance 
indicated that this difference was reliable at the . 01 
level[ F(1, 58)-23.40] . Prepositions are essentially 
relater words which signal a relationship of depend- 
ency for the objectof the preposition with respect to 
the item modified. As such, prepositions are often 
uniquely determined by the context. Therefore 
Science passages having more prepositions, are 
easier for the Ss when comprehension is tested by 
the cloze procedure. Further inspection of the pas- 
sages of these two Content Areas suggested that T. V .- - 
Movies-- Theatre passages were generally written in 
more colloquial language and contained more proper 
nouns and compounds. 


DISCUSSION 


The result that there is a reliable difference in 
cloze performance betweenthe High level of I-suffix 
density and both Medium and Low levels suggests 
that passages in content areas which normally con 
tain a high proportion of L-suffixes should be more 
difficult inter ms of cloze performance than passages 
in content areas in which few L-suffixes normally 
appear. Preliminary analyses of Content Areas in- 
dicated that Science passages from newspapers con- 
tain significantly more L-suffixes than either of the 
other two Content Areas under investigation. The 
fact that Science passages give the highest cloze per- 
formance when L-suffixes are controlled suggests 
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that the proportion of L-suffixes isareasonably 
powerful predictor of cloze difficulty. However, the 
fact that cloze performance was not differentially af- 
fected by the Low and Medium levels suggests that a 
rather high level of L-suffixes is necessary before 
this variable appreciably affects cloze Scores. It 
must alsobe pointed out that as a variable, L-suffix 
density covaries with severalother variables. Words 
Containing L-suffixes tend to be longer words than 


non-L-suffixed words and typically less frequent or 
familiar in the language. 


TABLE 5 


CORRELATIONS BETWEEN CLOZE SCORES AND 
SRAT SCORES AT EACH LEVEL OF LATIN- 
SUFFIX DENSITY AND AT EACH CONTENT AREA 


SSS 


Treatment 


Correlation 
SSS Se On 
Science .515 
Human Interest ‚612 
T. V. --Movies--Theatre . 644 
High . 508 
Medium ‚651 
Low + 610 


The reliable superiority of cloze 


performance on 
Science to that of T.V.--Movies-- 


Theatre seems to 


These factors 
educing context clues and 
ic redundancies, 


Results of the correlational analyses between 


FOOTNOTE 


This research was supported pursuant to a con- 
tract with the United States Department of 
Health, Education and Welfare, Office of 
Education, 
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ABSTRACT 


Educators disagree on the relative merits of stating classroomobjectivesbehaviorally or nonbehaviorally 


and have done little to add data to their argument. 


An experiment was conducted in the field of social science 


where one of three lists of objectives--one list nonbehavioral, the other two behavioral--was randomly assigned 
to participating high school social studies teachers who were instructed to teach objectives in their classes. Unit 
sampling was used and eighteen classrooms were involved. Students were measured, using a form of item sam- 
pling, onthe acquisition of the five skills stated in the behavioral objectives as well as on eighteen transfer skills. 
Teachers' faulty understanding of objectives, indicated by their inability to provide relevant classroom practice 
and to identify, when asked, test items measuring given objectives, may have accounted for lack of differences. 


THERE IS widespread interest in the educational 
worldregarding how one might best state objectives 
for instructionaleffectiveness. The controversy 
centers about the use of objectives stated in opera- 
tionalterms, specified with respect to the behaviors 
the student will be able to demonstrate after instruc- 
tion. Contestants have argued with varying degrees 
of eloquence either favoring or opposing the precise 
explication of objectives, Proponents cite the obvi- 
ous value іп ће teacher’s knowing where he is going 
and how to evaluate when he gets there (4, 6, 9, 11). 
Others (1, 2, 3, 8) worry that precise objectives 
hinder the full development of the student and force 
the teacher into an inflexible mode. The argument 
is ceasing tobe academic with the advent of the Pro- 
gram Planning and Budgeting System (PPBS) in school 
finance and the dependence of such a system on oper- 
ationally defined and measureable effects of instruc- 
tionally related expenditures. 


Unfortunately, there has been little concrete re- 
Search conducted into the effects on students when 
teachers use behavioral or nonbehavioral objectives. 
McNeil (10) investigated the effects achieved when 
Student teachers were told that their grade would be 

‘eter mined by the effects they produced in their learn- 
ers rather than the usual means- referenced observa- 
oH. Scales, His resultsfavoredteachersin the ends- 
bonne’ rather than means-oriented groups, but 
ctugeS® the results were related primarily to the 
сет teachers’ motivation for a grade, the findings 


оа 1 А n 
s rover ee applicable to the behavioral objectives 


Perhaps the major difficulty with empirical investi- 
gation has been in conceiving of an adequate design 
for anadequate experiment. Because most teachers 
have disparate objectives, comparisons between the 
effects they produce in their students are difficult, 
Problems are encountered if one randomly provides 
one group of teachers with a set of behavioral objec- 
tives and another group with nonbehavioral objectives 
and observes the effects on the students’ performance 
of the behavioral objectives. Sucha model is clearly 
biased against the teachers who received nonbehav- 
ioral objectives since they might have thought of ob- 
Servable ways to test the nonbehavioral objectives 
other than those hit upon by the researcher, 


The research reported herein sought to eliminate 
this design problem by allowing the comparison be- 
tween the effects of teachers’ use of behavioral and 
nonbehavioral objectives to be mate on criterion 
measures which took account of interpretations of the 
nonbehavioral objective other than the specific behav 
ioral objectives used in the investigation. 


SUBJECT AND SAMPLING 


Subjects were eighteen experienced teachers of 
social studies in Nevada secondary schools, These 
teachers were urged to participate in the evaluation 
of anew instructional unit by their principals, Teachers 
were assignedat random to one of three treatment 
groups. 
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MATERIALS 


The instructional materials used in the experiment 
consisted of a resource unit in the fieldof social 
Science research methodology. The topic of the unit 
was chosen because it could fit in anywhere in the 
Curriculum and would be relatively free of the con- 
tamination of previous instruction, since the topic 
is not typically in the curriculum. The unit consis- 
ted of a list of five objectives, expository material 
for the teachers' use to refresh themselves in the 
Subject matter, possible activities, and abstracts 
of actual investigations conducted in the broad field 
of social science. The activities were constructed 

Sothat some were clearly relevant to the behavioral 
Objectives and some were irrelevant to those 
objectives, 


TREATMENT 


The design of this Study attempted to deal with 
the problem of multiple interpretations of nonbehav- 


Instead of comparing 
Student performance only onthe particular behavioral 


e made by four grad- 
- For example, one 
Understand The 


d Causal Hypothe- 
Ses. Four different behavioral objectives dealing 


with such “understanding” were constructed, 


Two examples were: 


1. Whengiven a hypothesis the Student can ident- 


ify it as either descriptive or causal, 


2. Student will be abl 
trasting the use of 
potheses, 


е to write a sentence con- 
descriptive and causal hy- 


Teachers in both “behavioral objective’’ 
received only one of the behavior. 
to teach, but students in all group: 
on their achievement of all such o 
procedure takes into account at le 


ferent interpretations of a given n 
tive, 


treatments 
‘al sub-objectives 
з were measured 
bjectives. Such a 
ast some of the dif- 
onbehavioral objec- 


Inthis study, 


five nonbehavioral objectives on five 
distinct topies 


in social science research methods 
were written, For each of the nonbehavioral objec- 
tives several different behaviors were described 
which might be reasonable interpretations of the non- 
behavioral objectives. Two of the nonbehavioral ob- 
jectives had four behavioral interpretations, and the 
other three nonbehavioral had five behaviors each, 
resulting ina list of twenty-three behavioral objectives, 


a representative list of five behavioral obj Gn hae 
to select randomly from within each topic SEHE 
tivetobeused. As insurance against the e 
thatthis single random collection of behavioral ol ө 
tives was somehow typical of peculiar by e 
another set of five behavioral objectives MR 
by asking instructional experts to select withi 2. 
of the five categories the one behavioral object ted 
they felt would facilitate learning of all other че" 
tives within the same category. Experts were aske 
to rank each objective within each topic area and in 
each case the modal rank was selected. 


Eachtreatment, then consisted of a separate и 
of five objectives. Each Objective matched with о А 
jectives onthe other lists in terms of content (О 
tive one on all three lists, for example, dealt eni a 
hypotheses); however, one list was composed of be 
havioral objectives selected at random (RBO) the 
second list was composed of behavioral Cae 
selected by instruction experts (EBO), and the las 


list was composed of the nonbehavioral objectives 
(NBO). 


INSTRUMENTATION 


Test items f 


Objectives were constructed. All items were item, 
analyzed based on trials with the material with fifty 
Seven high school students. The Kuder Richardson 


Item sampling (5 and 7), an approach to gne 
many observations ina relatively limited time period: 
was employed since a representative test measuring 
all twenty-three Sub-objectives would require {00 
much time to administer in a normal school petting: 
Item sampling in this case entailed giving differen 
short but comparable forms of the test to learner? 
Within a given classroom, Four forms of the ead 
were used so that the tesing period could be conte. 
in oneclass period, While some objectives Em. 
adequately measured by one item (‘Describe the m 
ference between causal and descriptive hypotheses 1 
other objectives require morethan one item (given 
а hypothesis, label it as either causal or descriptive 
hypotheses"), However, the test items were randon 
ly assigned to the four test forms within each of E. 
twenty-three behavioralobjectives, sothat each un 
contained the same number of items measuring а gu. 
Objective as any other form, The test forms Wer 
non-overlapping, except for those objectives for WRIT 
only one item could sensibly be constructed. rs 
visions were made so that tests were randomly en 
Signed within each classroom, which resulted in 0n 
quarter of the Students in a given classroom taking 
any one form of the test. While different learners 
within a classroom received one of four versions "e 
the posttest, the total effect was that each teacher Е 
class received allitems devised for the twenty-thre 
behavioral objectives in the same proportion as any 
other class, Therefore, while individual students 
performances within and between classes were nO 
necessarily comparable, the teachers’ instruction? 


n 


„ 
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effectiveness, as gauged by the mean performance of 
the class, could be compared. 


PROCEDURE 


The eighteen teachers had been assigned at ran- 
dom to one of the three treatments. Each teacher 
received a list of five objectives, the resource unit, 
a sealed test packet and a questionnaire. Teachers 
in the nonbehavioral objectives (NBO) groupreceived 
five objectives which specified the content areas 
clearly but did not describe the specific behaviors to 
be tested. Teachers in the random behavioral objec- 
tives (RBO) group received a list of five objectives, 
selected at random from within each of the five ma- 
jor content areas froma list of twenty-three specific 
objectives. The instructional expert behavioral ob- 
jectives (EBO) group received a list of five behavioral 
objectives selected from the twenty-three objectives 
by a teamof individuals reputedly expert in the field 
of instruction. 


Teachers were told that they were helping to as- 
sess the usefulness of the objectives and resource 
materials with their students and that results by teach- 
er would not be given to the district or supervisors. 
Teachers were instructed to teachthe five objectives 
within twoclass periods (or taking about 100 minutes 
if their classes were on flexible scheduling basis). 
The tests were administered immediately following 
instruction. Eachtest packet containedthe four test 
forms in random order. The teachers were asked 
to distribute the tests іп that order as they removed 
them fromthe sealed packet. Teachers were asked 
to return to questionnaires and tests within one week 
of completing the unit. 


ANALYSES AND RESULTS 


The students’ tests were scored and weighted so 
that each of the twenty-three objectives made an 
equal contribution. A comparison among all three 
groups on the total test score, reflecting twenty- 
three objectives, was conducted through use of one- 
way analysisof variance. Separate analyses of vari- 
ance were conducted for the three groups by each of 
the five topic areas. Analyses were also conducted 
comparing the random behavioral group and the non- 
behavioral group and between the expert behavioral 
groupand the nonbehavioral groupon their respective 
lists of objectives. Results of the analyses yielded 
no significant differences. The data of real interest, 
however, were gathered from the teacher question- 
naire. Teachers were asked to select test items 
which directly measured the objectives they had been 
given, It was believed that a teacher who carefully 
understood the objectives should be able to match 75 
Percent of items with their proper objectives. In 
Table 1 is the average number of correct responses 
made by teachers in the behavioral and expert behav- 
loral groups, 


t As can be seen, most teachers were responding 
0 these binary choices at approximately chance levels 
Percent), Such information is extremely impor- 
. when considering the implications of the pupil 
achievement data, 


T я 
зве Questionnaire also called for the teacher to 
ted to the type of activities he used and time alloca- 
each. Activities chosen or described were 


TABLE 1 


MEAN AND STANDARD DEVIATION FOR SCORES 
FOR MATCHING 12 ITEMS WITH GIVEN BEHAV- 
IORAL OBJECTIVES 


Sample Size 13 
Mean 6.12 
Standard Deviation 7T 


judged either appropriate to the Objective or inappro- 
priate and a percentage of appropriate activity time 
compared with overall time was computed for each 
teacher in both the random behavioral and expert be- 
havioralgroups. No such percentage was computed 
for the nonbehavioral teachers since, given nonbe- 
havioralobjectives, one might infer that almost any 
activity dealing with the content area would be appro- 
priate tothe objective. Percentagesfor the random 
behavioral and expert behavioral groups are present- 
ed in Tables 2 and 3. 


TABLE 2 


PERCENTAGE OF INSTRUCTION TIME JUDGED 
TO BE ALLOCATED TO RELEVANT ACTIVITIES 
BY BEHAVIORAL GROUPS 


Sample Size 13 
Mean ‚2195 


Standard Deviation .3458 


TABLE 3 


PROPORTION OF TOTAL NUMBER OF REPORTED 
ACTIVITIES JUDGED TO RELEVANT TO GIVEN 
OBJECTIVES 


Sample Size 13 
Mean .2129 


Standard Deviation .3112 


DISCUSSION 


The retrospective explanations reasonably justi- 
fied by this configuration of data are straightforward, 
A major inference one must make is that the treat- 
ment variations as presented were not powerful enough 


8 


to modity the dependentvariable performance. This 
is reflected in the within group variability of the 
teachers. However, thetreatment in the behavioral 

objective groups did not “аке” as expected. For 

example,one wouldimaginethatateacher who really 
attended to the precise nature of the objectives (be- 
havior and content) would be able to identify those 

test items which were clear measures of the objec- 
tives, The data reveal that this was not the case in 

both behavioral groups. 


Further support for the contention that the behav- 
ioral objectives treatment had no major impact on 
the teachers is supplied by the data obtained from 
teacher activity reports, where the teacher was sup- 
posed to indicate which activities he selected during 
the instruction. Estimated time devoted to each ac- 
tivity was also reported. Proportions were compiled 
for time spent in activities over total time reported 
and for the number of activities judged relevant (by 
trained evaluators) over total number of activities 
reported. The figure for the behavioral groups on 
the time percentage was approximately , 28, On the 


number of relevant activities proportion the groups 
averaged . 27. 


If one believes 
make a 


cause them to change 
fficient, instructional 
Research into the application of suitable 


y models for reinforcing measurable 
changes in student behavior is essential, 


FOOTNOTE 


l. The materials used inthis research were developed 
in the course of Project No, 5-0566-2-12-1 
Supported by the Office of Education,U.S. De- 
partment of Health, Education and Welfare, 
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ABSTRACT 


The goals of this study were to determine the effectiveness of the minicourse model in changing student 
teacher behavior and to estimate the effects of the microteaching format and use of videotape feedback within this 
model, Feedback and practice in the microteaching format were manipulated with four groups of student teach- 


ers while a fifth group served as a control. 


Eleven behaviors were scored for the four treatment groups on precourse and post-course videotapes. 
Eighteen of the resulting comparisons were significant beyond the . 01 levelandthree beyond the . 05 level in the 


desired direction. Three of the elevencontrol group gains were significant at the . 05 level. 


Treatment groups 


that did not receive videotape feedback and did not practice in the microteaching format were not significantly 


different than groups that did. 


THE MINICOURSE MODEL 


SINCE March, 1967, the primary program of the 
Far West Laboratory for Educational Research and 
Development has been aimed at developing and test- 
ing the minicourse instructional model as a tool for 
changing the classroom behavior of teachers, The 
minicourse model was initially based upon the micro- 
teaching technique used at Stanford University in the 
training of teacher interns. Microteaching as devel- 
oped at Stanford has several characteristics: 


1. First, a set of specific teaching skills is 
studied by the intern. 


2. Then, the intern attempts to apply the skills 
in a short lesson, usually 5 to 10 minutes , 
with four or five pupils. 


3. This lesson is recorded on videotape and im- 
mediately after its completion, the intern 
watches a replay of the lesson. 


а. During the replay а specially trained super- 
visor gives the intern specific feedback on his 
Performance in the skills. 


The intern then replans the lesson and reteach- 
es it to another group of four or five pupils. 


The mis; 
int, ™inicourse model differs from the Stanford 


nm ү 
im шерш уен ways which generally reflect 
Era; situation found in iversi service 
m às Opposed to an inse Evi E BED anaes 


be offered in any school district. One major differ- 
ence is that the minicourse model provides a se 1f= 
contained package of inservice training materials that 
canbe used in any school where videotape recording 
system is available. The second major difference is 
that while the Stanford Intern Program employs feed- 
back from trained supervisors, the minicourse pro- 
gram attempts to provide feedback through carefully 
structured teacher self-evaluation of televised lesson 
replays. The minicourse also relies heavily upon 
filmed illustrations by model teachers rather than 
supervisory feedback to provide the trainee with a 
basis for discriminating the behavior patterns or skills 

to be learned. Some research evidence raises seri- 
ous doubts about the value of supervisory feedback, 
Acheson (1), in a study of sixty teaching interns at 
Stanford, found that television feedback with a super- 
visor conference was no more effective than television 
feedback without a supervisor conference. Further- 
more, a supervisor conference without television feed- 
back did not produce greater behavior changes than 
occurred with a control group which received neither 
television feedback nor conference. Ina study of the 
effects of student versus supervisor feedback on teach- 
ing methods, Tuckman and Oliver (12) found that 
supervisor feedback resulted in а negative shift in 
teacher behavior, i.e., away from the direction sug- 
gested by the supervisor. 


On the other hand, evidence on the use of television 
replays as a source of feedback, such as Acheson's 
(1) study, is generally positive. Orme’s (11) study of 
perceptual modeling versus symbolic modeling is also 
pertinent to the minicourse instructional model, In 
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symbolic modeling the subject was told what to do 
while in perceptual modeling, he was shown a filmed 
model who portrayed the desired behavior. Perceptual 
modeling consistently brought about greater changes 
in the behavior of teaching interns than symbolic mod- 
eling. Using models in place of Supervisory feed- 
back also has the practical advantage of making it 
possible for the minicourse to be developed as acom- 
plete self-contained package which requires no Spe- 
cial skills to administer. 


PROBLEM 


The major goals of this Study were to estimate 
the degree to which (a) practice in the microteaching 
format (i.e., short lessons and few pupils) and (b) 
feedback from the videotape replay influenced learn- 
ing in the minicourse model, A third goal was to 
determine the effectiveness of the minicourse as a 
technique for changing the behavior of student teachers, 


The specific hypotheses that were tested in this 
study are: 


1. Student teachers completing the entire Mini- 
course 1 will display significantly greater 
changes in the teaching behaviors covered in 
Minicourse 1 than a Similar group which re- 
ceives the entire course except for practice 

in the microteaching format and VTR feedback, 


2. Student teachers completing the entire Mini- 
course 1 sequence will display significantly 


3. Student teachers taki 
junction with student 
icantly greater chang 
lors covered in Mini 
jects taking student 
minicourse, 


ng the minicourse in con- 
teaching will make signif- 
es in the teaching behav- 

course 1 than similar Sub- 
teaching without the 


advantages over practice in thi 
Bush and Allen (6) 


although the videotape replay appears to be an inpor 
tant factor in the minicourse model, there is no di 
rect evidence on the amount it contributes to the 
behavioral changes brought about by this model. Since 
many schools do not have access toa videotape 
recording system, it is important to learn whether ~ 
the minicourse model can bring about significant be 
havior changes when used without the videotape re 
corder (VTR). 


PROCEDURE 


Developing Minicourse 1 


The development cycle employed at the Labora- 
tory calls for three field tests of each minicourse, н 
each followed by a revision based on field test infor: 
mation. The preliminary inservice form of Man 
course 1 was developed in the spring of 1967 e 5 
field tested in July of that year, In the Laboratory 
Rand D cycle, the purpose of the preliminary fie 
test is to get qualitative feedback from Laboratory 
Personnel who conduct the test and from participating 
teachers. Based on the results of this field test, the к 
course was revised and a main field test was gonane 
ed in October, This test was designed to determin 
if the minicourse brought about significant changes 
in the skills and behaviors covered in the course: 
The performance of the forty-eight participating 
teachers was videotaped in 20-minute priescours 
and post-course lessons recorded under essential у 
the same conditions, The tapes were thenassigne 
randomly to trained raters who replayed the toi 
and scored them on the behaviors taught in the gour 
Raters were not regular Laboratory employees, н 
по stake in the Success or failure of the program 
and did not know Whether any given tape was made 
before or after the course. Main field test results 
on the inservice teacher sample were reported үт 
previous paper (4), Briefly, it was found that 0! : 
thirteen scores obtained from the pre-course ап 
Post-course tapes, eleven showed statistically o 
nificant gains. Some of the gains were not only E 
tistically significant, but were large enough to ARE 
Best a major change in the way the participati т 
teachers conducted their discussion lessons. alk 
example, the average percentage of teacher Е 
during class discussion changed from 51.64 pere, 


on the pre-course tapes to 27.75 percent on the 
course tapes, 


Minor rey: 
test and a fin 
February 
could be conducted op 
mum of Laboratory help, i.e., was it ready for ess" 
ational use? i 1 core 
ful, and only minor revisions were required T 
the course was made available to the schools 
Operational use in April, 1968, 


was 


The operational test form of Minicourse ted in 


adapted for preservice use in the study герог 
this paper, 


THE TREATMENT 


Minicour 
Vice course 
goal of the c. 
room beha 


mere 
Se 1 was initially developed as an ins ne 
for teachers in grades 4, 5, and | | clas" 
ourse is to change twelve specific d o! 
Viors related to the teacher's metho 
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conducting a discussion lesson. These twelve behav- 
iors are organized into four instructional sequences. 
The objectives of each instructional sequence and the 
three behaviors covered in each are given in Table 
l. An instructional sequence in Minicourse 1 requires 
three 75-minute periods to complete. During the 
first period, the student teacher subgroups that were 
given the entire course started by watching the in- 
Structional lesson, a 20- to 30-minute motion ріс- 
ture. In this film three behaviors are described by 
à narrator and illustrated with film clips which show 
teachers using the behaviors. A rationale for using 
each behavior is also given although this is treated 
in more depth in a handbook given the student teach- 
er at the start of the course. Immediately after 
Watching the instructional film, the student teacher 
watched a model film in which a model teacher illus- 
trates all three of the behaviors in a microteaching 
situation, i.e., a lesson 5 to 10 minutes long involv- 
ing four to eight pupils. On the first showing of the 
model lesson, the student teacher was asked to iden- 
tify each of the three behaviors on a checklist as it 
Occurred. A second form of the model lesson was 
then shown in which the narrator named each behav- 
lor as it occurred so that the student teacher could 
Check his ability to recognize the behaviors. After 
viewing the model film the student teacher was given 
instructions for preparing a short discussion lesson 
in which he could practice the three behaviors de - 
Scribed in the instructional and model films, 


During the second period in the instructional se- 
quence the student teacher took four to eight pupils 
from his assigned class to the microteaching room, 
started the videotape recorder and taught the lesson 
he had planned. When he had finished teaching, he 
sent the pupils back to the regular classroom and 
replayed twice the videotape recording of his lesson. 
For each of these replays, an observation form was 
provided which focuses the student teacher's atten- 
tion on the three specific behaviors he was instruct- 
ed to practice. After viewing the replays of his les- 
son the student teacher was instructed to replan the 


same lesson so as to make more effective use of the 
behaviors practiced. 


At the start of the third session, the student teach- 
er again went to the microteaching room and taught 
his revised lesson to a different group of pupils from 
his assigned class, The lesson was again recorded 
on videotape which the student teacher replayed twice, 
each time using a different observation form that he 
focuses on a specific teaching behavior. 


Having completed the first instructional sequence, 
€ was then ready to proceed to the next sequence in 
Which he followed the same pattern of viewing instruc- 
tional and model films, microteaching and replaying 
а lesson, replanning the lesson, and reteaching and 
replaying the revised lesson. 


SUBJECTS 


In this Study, student teachers from three teacher 
College a titutions were employed as subjects. In 
.» Seventeen student teachers completed the 
tion (Grou Qurse 1 as described in the previous sec- 
"P А-1), A second group of sixteen student 
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TABLE 1 


ACH IN- 
OBJECTIVES AND BEHAVIORS FOR E. 
STRUCTIONALSEQUENCE IN MINICOURSE 1 


Instructional Sequence I: 


jecti i that will 
Objective: To change teacher behaviors ^ 
increase the pupil's readiness to respond to dis- 
cussion questions. 


Specific behaviors covered: 

A. Ask question, pause 5 seconds, then call on 
pupil. i 

B. Deal with incorrect answers in an accepting, 
nonpunitive manner. 

C. Call on both volunteers and non-volunteers. 


Instructional Sequence II: 


Objective: To change teacher behavior so as to 
decrease teacher participation and raise the level 
and amount of pupil participation. 


Specific behaviors covered: 

A. Redirection-directing the same question to 
several pupils. 

B. Framing questions that call for longer pupil 
responses and preclude one word replies. 

C. Framing questions that require the pupil to 
use higher cognitive processes. 


Instructional Sequence III: 


Objective: To increase the teacher's use of prob- 
ing behaviors in order to guide the pupil to more 
complete and thoughtful responses. 


Specific behaviors covered: 
A. Prompting. 

B. Seeking further clarification and pupil insight 
C. Refocusing the pupil's response. : 


Instructional Sequence IV: 


Objective: To reduce the frequency of teacher be- 


haviors that interfere with the flow of the discus- 
sion. 


Specific behaviors covered: 

A. Teacher should not repeat her questions, 

B. Teacher should not answer her own questions 
C. Teacher should not repeat pupil answers, ` 


teachers (Group А-2) were given the course hand 
shown the instructional and model films, and we 
instructed to practice the behaviors in their regular 
student teaching classrooms. This groupdid not 
practice in a microteaching situation and did not view 
replays of their performance. 


book, 
re 


In College B, fifteen student teachers completed 
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the entire Minicourse 1 (Group B-1). A group of 
Seventeen student teachers were given the entire 
course except for the videotape recording and replay 
(Group B-2). This group received the handbook, was 
shown the instructional and model lesson, and prac- 
ticed in a microteaching situation. They also evalu- 
ated their microteaching lessons but since they did 
not view themselves on videotape, the evaluation 
forms developed for this group were less specific. 


In College C, a group of student teachers taking 
Minicourse 1 were to be compared with а control 
group which received regular student teaching only. 
However, a sufficient number of student teachers 
could not be assigned to the two treatments so itwas 
decided to collect pre- and post-course videotapes 
on the control group only, and employ this as a com- 
parison group for the other two colleges (Group C- 1). 


Tn Colleges A and B student teachers scheduled 
to teach in grades 4, 5, and 6 could not be assigned 
to the treatments in a completely random fashion. In 
College A, those student teachers participated who 
had been enrolled with four college supervisors. Stu- 
dent teachers were assigned arbitrarily to su per- 
visors. Whichtreatment the studentteacher received 
was determined by the school to which he was as- 
signed for student teaching--another arbitrary admin- 
istrative decision. 


College B student teachers had received their stu- 
dent teaching assignments before arrangements were 
made to participate in the study. Student teachers 
in schools which had four student teachers assigned 
to grades 4, 5 or 6 were placed in Group B-1. Stu- 
dent teachers in schools which had two or three as- 
signed to grades 4, 5 or 6 were placed in Group B-2, 
This decision was made to get maximum use out of 
the video equipment, since up to four teachers in a 
given school can use the same set of equipment, No 
variables that would bias the assignment of student 
teachers to the treatments appeared to be Operating 
ineither college. In College C, all student teachers 
assigned to five elementary schools were assigned 
to the comparison group. Again, no apparent bias 
was operating in this selection. 


The dependent variable in this study was the sub- 
ject's performance on two 15-minute videotapes re- 
corded in a regular classroom with the studentteach- 
er's entire class; one immediately before and one 
immediately after the experimental treatment, A 
week before the pre-course tape was recorded and 
again before the post-course tape was recorded, sub- 
jects were given instructions on how to preparetheir 
discussion lessons for videotape recording. During 
the recording the camera was placed in the class- 
room in a position that would provide a good view 
of the student teachers and pick up at least half ofthe 
pupils. Four microphones were positioned around 
the classroom. At College A the videotape recorder, 
monitor and operator were positioned outside of the 
classroom, while in College B they were positioned 
at the rear of the room. The student teachers were 


given a brief warm-up period before the recording 
was started. 


There seems little doubt that the presence of the 
camera and microphones changes the classroom situ- 
ation. However, there is some evidence that this 
change is small after pupils have had a few minutes 


to adjust to the presence of the equipment (9). Inthis 
study an effort was made to minimize this influence 
and hold it constant for all subjects in a given college. 
Other alternatives, such as using a hidden camera 

or leaving the camera in the room for several days 
and recording on a time sampling basis, were consid- 
ered, but could not be used because of limitations in 
the equipment and classrooms that were available. 


After the experimental treatment, the pre-course 
and post-course tapes were coded sothe raters would 
not know when a given tape had been recorded. Raters 
were trained to reliably score two or three specific 
behaviors ata time. During the training sessions, 

a very precise operational definition of each behav- 
ior was developed and decisions were made on the 
Scoring of borderline cases. The fifteen raters were 
then assigned randomly to three groups. Eachgroup 
was assigned five tapes, and raters in the group in- 
dependently scored these tapes. The average inter- 
rater reliability was then computed, Table 2 gives 


the average correlations among the five raters in 
each group. 


The factor that appeared to have the greatest in- 
fluence upon the reliability coefficient was the fre- 
quency of occurrence of the behavior. Reliability 
coefficients for behaviors that occurred with low 
frequency such as answering own questions, prompt- 
ing, and asking higher Cognitive questions, were 
lower and less stable from group to group than for 
high frequency behaviors such as redirection and re^ 
peating pupil answers. The specificity of the behav- 
ior also influenced reliability since it is much easier 
for an observer to determine whether the teacher has 
repeated a pupil answer than to decide whether а 
siven question requires the pupil to use higher cog- 
nitive processes, Scoring of an earlier set of tapes 
had indicated that four of the Scores were so specific 
that virtually no differences occurred among raters. 
Two of these, length of teacher pause and percentage 
of teacher talk, were obtained by timing teacher 
pauses and talk with a stop watch. The other two, 
number of words per pupil response and number of 
one word responses required a simple word count of 
each pupil response. 


TABLE 2 


AVERAGE RELIABILIT EE GROUPS OF 
RATERS Y FOR THR 


Å I Immm 


oup 
Behavior 2 as я £ 
1. Redirection .94 ida a 
2. Number of higher .62 d ў 
Cognitive questions 
: Number of fact questions .87 288 d 
- Prompting .65 98 : 
5. Seeking further clarifi- .94 #98 É 
cation 
т Repeat Questions 85 91 i^ 
8. nswer own questions 75 ae е 


Repeat pupil answers .995  .99 .98 


Once the initial fi he 

ini five tapes had been scored, th® 

ire WE tapes were designed randomly to the rat 
Я en all tapes were scored for the first S° 


| 


J 
i 
П 
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of behaviors, raters were trained to score the next 
set. This cycle was repeated until all tapes had 
been scored for all behaviors. 


RESULTS 


One of the twelve behaviors covered inthecourse, 
**calling on both volunteers and nonvolunteers’’ was 
found to be unscorable. In many classrooms it was 
not possible to position the camera so all pupils 
were in camera range, and it was necessary to see 
the pupil in order to determine whether or not he had 
volunteered. Two other behaviors, ‘‘dealing with 
incorrect answer in an accepting, non-punitive man- 
ner,” and *'refocusing the pupil's response," were 


E 10usly scored (4) showed virtually no variation in 


* pet scored because the inservice field test tapes pre- 
S des behaviors. Teachers almost never used a pu- 


nitive manner in dealing with incorrect answers; 
perhaps because of the presence of the video camera. 
Refocusing also appeared so rarely in either pre- 
course or post-course tapes that scoring did not 
appear worth the equipment and rater time involved. 


A total of eleven scores were obtained from each 
tape. Scores were obtained on nine of the behaviors 
listed in Table 1, and in the case of one of these, 
‘framing questions that call for longer pupil re- 
sponses,’’ two scores were obtained, The percent- 
age of teacher talk was obtained by timing pupil and 
teacher talk with a stop watch, Although not one of 
the specific behaviors taught, this percentage is an 
important variable in class discussion and relates 
to one of the course objectives. 


Table 3 gives the pre-course and post-course 
mean scores for the five groups as well as t ratios 
between pre and post tape scores for each group. The 
one-tailed test and the .05 confidence interval were 
applied in testing the hypotheses, 


The reader will recall that Groups A-1 and B-1 
were given the entire Minicourse 1; Group A-2 re- 
ceived the entire course except microteaching and 
videotape feedback, Group B-2 received the entire 
course except videotape feedback, and Group C-1re- 
ceived none of the minicourse materials. All sub- 
jects were enrolled in student teaching. 


The first behavior that Minicourse 1 attempts to 
change is the length of the student teacher's pause 
between framing his question and calling upon a pu- 
pilto respond. The course attempts to train teach- 
ers to pause from 3 to 5 seconds after asking a ques- 
tion and before calling on a pupil. The rationale for 
this behavior is that such a pause gives pupils more 
time to frame a thoughtful response to the question. 
Also, since a pupil’s name is not called until after 
the pause, there is some pressure on all pupils to 
plan a response in case they are called on. In con- 
trast, calling the pupil's name before or immediate- 
ly after framing the question immediately notifies 
other pupils that they are not going to be called upon 
to respond so they are under no pressure to think 
through an answer. Of the five groups, only Group 
B-1 made a significant increase in the average length 
of this pause, 


The course also attempts to increase the number 
of times student teachers use redirection in theclass 
discussion situation. Redirection is the technique 


of framing questions in such a way that the question 
can be directed to several pupils rather than to a 
single pupil. The teacher asks the question and re- 
directs it to a number of pupils, each of whom con- 
tributes to a complete answer. Redirection has the 
advantage of increasing pupil participation and often 
leads to direct interaction among pupils in the dis- 
cussion situation. All groups except C-1 made some 
improvement in redirection with this improvement 
reaching statistical significance for groups A-2and 
B-1. 


Another objective of the course was to train stu- 
dent teachers to ask questions that call for longer 
pupil responses and to ask fewer questions that can 
be answered yes or no or with a single word. A word 
count of pupil responses was made on each tape to 
determine the average length. It will be noted that 
the average length of pupil replies on the pre-course 
tapes ranged from six to seven and a half words. Vir- 
tually no change occurred for Group A-2 and Group 
C between pre-course and post-course tapes. How- 
ever, significant gains bringing the average length 
of the pupil reply beyond ten words were made by 
Groups А-1, В-1 and B-2. These data are supported 
by previous research on inservice teacher training 
in which the average length of pupil response was 
found to increase from 5.63 words to 11.78 words for 
inservice teachers who were given the complete Mini- 
course 1 (4). The number of one-word replies was 
also tallied for 5-minute samples of pre-course and 
post-course tapes. Groups B-1, B-2, and C allmade 
significant reductions on this variable. 


The course also attempted to train student teach- 
ers to ask questions that required pupils to use high- 
er cognitive processes in framing their replies. All 
teacher questions were classified as either fact ques- 
tions, higher cognitive questions, or proceduralques- 
tions. The percentages given in Table 3 indicate the 
proportion of higher cognitive questions to the total 
of higher cognitive and fact questions combined, It 
will be noted that there are no significant gains inthe 
percentage of higher cognitive questions and, in fact, 
a significant loss occurred for Group А-1. In re- 
examining Group A-1 tapes, it appears that this loss 
is largely an artifact of the system used for classity- 
ing questions. Several student teachers in this group 
asked questions which called for opinions rather than 
higher cognitive processes. These were classified 
as higher cognitive questions, however, on the classi- 
fication system used. For example, a question such 
as ‘‘How did you like this poem?" is technically an 
evaluation question although such questions usually 
elicit replies that do not appear to call for higher cog- 
nitive processes. The experienced teachers inthe 
previous inservice sample made less extensive use 
of opinion questions so that the very simple classifi- 
cation System used seemed adequate, However, in 
a similar study using teacher interns, Claus (7) founa 
it necessary to set upa Separate questioning category 
for opinion questions which she included along with 
fact questions and labeled as lower order, Since fund: 
were not available to rescore higher cognitive ques- А 
tions, it will be necessary to await the results of fur- 


However, it 
Out signifi- 
ive questions 
© shown upin 


seems doubtful that the course brought ab 
cant increases in the use of higher cogniti 
since such increases would probably hav 
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spite of the confounding brought about by inclusion of 
opinion questions in the higher cognitive category. 


Two probing techniques that teachers can use af- 
ter the pupil's initial response to a question in order 
to lead the pupil to a more adequate reply are cov- 
ered in Minicourse 1. These are prompting, in which 
the teacher gives the pupil clues or follows up his 
initial response with leading questions and further 
clarification, in which the teacher attempts to get 
the pupil to clarify, elaborate or explain his initial 
response. It may be seen in Table 3 that no signifi- 
cant increases in prompting occurred. Student 
teachers in Group A-1 made significant gains in 
their use of further clarification. 

2 

^ Minicourse 1 also attempts to reduce or eliminate 
the teacher's use of three negative behaviors. These 
behaviors are repeating the question, answering ones 
own question and repeating the pupifs answer. Many 
experienced teachers habitually repeat nearly all of 
their questions. For example, in the inservice sam- 
ple previously studied, teachers with an average of 

9 years experience repeated their questions nearly 
fourteen times in a 20-minute discussion lesson. In 
contrast, it will be noted that the average teacher 
in preservice training repeated questions five times 
or less in а 15-minute discussion lesson. Repeating 
one's questions is considered an undesirable behav- 
ior because it takes up valuable discussion time, Al- 
so, when used habitually, it conditions pupils not to 
listen to the first statement of the question. However, 
there are certainly situations in the typical lesson 
where it is desirable for the teacher to repeat aques- 
tion. Since few of the subjects involved in the cur- 
rent research habitually repeated their questions , 
their behavior for the most part was at an acceptable 
level prior to the experimental treatment. Two groups, 
B-2 and C showed significant decreases in this be- 
havior. For Group С this decrease must be attribu- 
ted to their intervening experience in student teach- 
ing. This was probably the main factor operating 
with Group B-2 as well. 


The disadvantages of the teacher answering his 
or her own questions are obvious. If carried to the 
extreme, this behavior results in the teacher giving 
a monologue rather than conducting a discussion les- 
son. In any case, it deprives pupils of the chance 
to participate in the discussion and increases the pro- 
portion of discussion time taken up with teacher talk. 
Data from our inservice sample indicated that few 
teachers answer their own questions with high fre- 
quency. This finding was supported in the preser- 
vice groups studied where the highest average fre- 
quency was three repetitions during a 15-minute 
lesson. However, in spite of the low initial level of 
this behavior, significant reductions occurred in 
Groups A-1, A-2, and B-2. 


Many teachers in the inservice study were found 
to repeat automatically all or most pupil answers. To 
be classified as a repetition in this study, the teacher 
had to repeat the pupil's answer nearly verbatim ad- 
ding no new ideas and making no major changes in 
the words used. Thus, restatement, elaboration or 
clarification of the pupil's answer was not classified 
as arepetition. It may be seen in Table 3 that on 
the pre-course videotapes, the student teacher groups 
repeated pupil answers an average of sixteen to 
twenty-three times in a 15-minute lesson. The 


frequency of this behavior was reduced significantly 
in all of the treatment groups. This would suggest 
that the minicourse, with or without microteaching 
and videotape feedback, can bring about significant 
reductions in the frequency of this behavior. It will 
be noted that virtually no change occurred in this be- 
havior for Group C-1. 


One of the objectives of Minicourse 1 is to reduce 
the proportion of time during the class discussion 
when the teacher is talking. Previous studies have 
shown that teachers talk as much as 70 percent of 
the class time, thereby severely restricting the 
amount of time available for pupil participation (8, 2). 
Data from the inservice field test of Minicourse 1 
revealed that prior to taking the course, teachers 
talked an average of 51.64 percent of the time during 
a class discussion period (4). It may be seen in Ta- 
ble 3 that student teachers also talk a considerable 
proportion of the time during class discussions. All 
four of the treatment groups dropped significantly in 
the percentage of teacher talk. In Group C-1 the 
percentage of teacher talk was also reduced signifi- 
cantly although a magnitude of this reduction was 
less than found in the treatment groups. This would 
suggest that the reduction of teacher talk in the four 
treatment groups was partially attributable to the stu- 
dent teaching experience and partially to the specific 
instruction of the minicourse. 


SUMMARY AND CONCLUSION 


The groups that completed the entire minicourse 
did not consistently make more or larger changes in 
behavior than the groups for which some element of 
the course was omitted. In College A, Group A-1 
made significant gains in the desired direction on 
five of the eleven scores as compared with four sig- 
nificant gains for Group A-2. However, analysis of 
covariance in which the final performance of student 
teachers in groups A-1 and A-2 was adjusted for 
their pre-course performance revealed only a signif- 
icant difference between the two groups. Group A-2 
made significantly greater changes in redirection 
than Group A-1 (F - 7.66). 


In College B, Group B-1, and B-2 both made sig- 
nificant gains in the desired direction in six of the 
eleven scores. Analysis of covariance revealed no 
significant differences in the gains made by the t wo 
groups. 


In College C, three statistically significant gains 
were made, These gains, however, were generally 
smaller than those of the treatment groups; none 
reaching the . 01 level of confidence. Thus, Hy pothe - 
ses 1 and 2 are not supported by the data. The omis- 
sion of videotape feedback and practice in the micro- 
teaching format from the minicourse model appears 
to have resulted in little or no redirection in the 


changes in teacher behavior brought about ini 
course 1. ы и 


Hypothesis 3 appears to be supported b 
The difference in performance Spent le au 
that completed the entire course, (Groups A-1 a сш 
B-1) on one hand and Group C- 1 on the Other, a А 
fairly large and favor Students who completed Mini $ = 
course 1. Analysis of covariance revealed that student 
teachers in groups А-1 and B- 1 made significanth " 
greater improvement than GroupC intwo scored: re- 
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peating pupilanswers(F=14.85), and percentage of 
teacher talk (F=5.69). 


Interview and questionnaire data obtained from the 
Subjects in the preservice study suggest a number 
of reasons for the lesser success of the course with 
these subjects. Probably most significant were the 
greater demands placed on the preservice teacher 
as compared with the inservice teacher. Student 
teachers had a great deal of difficulty carrying out 
allof the work demanded in the minicourse. Since 
the course was offered daily, it was necessary for 
Students to do some preparation each evening in or- 
der to be ready for the following day's work. This 
preparation time had to compete with assignments 
given students in their other classes and with de- 
mands made upon them by their supervising teachers 
and college student teaching Supervisors, In some 
cases, supervising teachers made demands that 
made it impossible for students in Groups A-1 and 
B-1 to carry out parts of the minicourse sequence. 
For example, a number of students were unable to 
complete all of their practice sessions in the micro- 
teaching setting because teachers would not release 
them at the scheduled time from their duties in the 
classroom, This experience suggests that in the pre- 
Service setting, the course should be offered on a 2 
or 3 day per week basis rather than daily. Super- 
vising teachers and other persons having control 
over the student teacher’s time should be very 
thoroughly briefed on the nature of the minicourse 
program and the importance of completing all of the 
required activities, 


One surprising outcome of the current study was 
the somewhat smaller behavioral changes brought 
about by Minicourse 1 in the presevice setting as 
compared with those found previously in field test- 
ing the course with experienced teachers. For the 
inservice sample, differences between pre-course 


A final conclusion might be that in s; - 
ber of problems that Fact. inc ie Me t 
1 in the two Colleges, several 
curred in the methods of quest; 
discussion lessons used by the 
teachers, 
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ABSTRACT 


The relationships between levels of cognitive functioning as suggested by the Taxonomy of Educational Ob- 
jectives, Handbook I and states of affective arousal as measured by Text Anxiety (the main variable of interest), 
General Anxiety, and Need for Achievement were investigated. Two separate samples were used: (1) 234 uni- 
versity students, and (2) 319 junior and senior class high school students. 


For the university sample significant negative correlations between Text Anxiety and Knowledge (recall of 
specific facts) and Test Anxiety and Comprehension were found. For the high school sample the negative corre- 


lation between Test Anxiety and Comprehension was significant. 


No significant relations between Test Anxiety 


and higher levels of cognitive performance were found. The results are discussed in terms of interfering effects 
to task performance that Test Anxiety may produce when the examinee is given no information to ‘‘workwith’’ 


and must rely essentially on memory. 


THE FACT that a relationship exists between af- 
fective states and cognitive functioning has been well 
established (6, 2, 10), Evidence indicates that anx- 
iety can act as a motivational force ontest perform- 
ance; but if the difficulty of the task is increased, 
those individuals with high test anxiety do not do as 
well as those with low test anxiety (5). Sarasonand 
others (9) have suggested that test anxiety seems to 
have an ‘‘interfering effect” and can act as an inter- 
vening variable on problem-solving performance. 


Sarason (8) demonstrated that highly test anxious 
individuals differ from those with low anxiety in that 
the former emit responses that are detrimental to 
the successful completion of the task, According to 
Sarason, high anxiety states manifest themselves in 
two ways: (a) self-verbalization (‘‘Ican’t pass this 
test’); and (b) autonomic responses which are 
Picked up by higher GSR ratings. Accordingly, high- 
ly anxious persons emit more interfering responses 
than do those individuals who exhibit less anxiety 
(4, 9). Certain levels of anxiety, however, can be 
optimal totest performance and levels either above 
or below this result in impaired test performance, 
Experimental support that the relationship between 
anxiety level and test performance can be described 
by an inverted U has been offered by Stennet (11) and 
Korchin (3), The variable of performance generally 
considered in anxiety-test performance relationships 
is level of task difficulty . 


Moving toa consideration of the interaction of lev- 
el of anxiety and performance with different problem 
types, Speilberger (10) reported that highly anxious 
subjects did as well as subjects who exhibited lower 
anxiety on clearly worded problems that suggested 
an answer rather quickly. Performance decrements 
however, for the highly anxious group were noted 
when the protocols were ambiguous or unclear, It 
was concluded that ‘‘task-produced interfering re- 
sponse tendencies interacted with highdrive to bring 
about the observed effects” (10:372), 

А In summary, there is evidence that levels of affec- 
tive arousal are related to the individual's ability to 
solve problems of various types and degrees of diffi- 
culty. The present investigation is concerned with 
further examination of such relationships, althou h 
irom a perspective that delineates test taking bebo he 
ior into specific levels of cognitive functioning T he 
Taxonomy of Educational Objectives (1) provide, 
rubrics for theclassification of cognitive functioni; 4 
їп a test situation that are (1) independent from ne 
another, and (2) representa hierarchy of Me 


PURPOSE 


The purpose of the present inves 
ascertain the relationships, ifany, 


and performance at different lev 
functioning. 


tigation was to 
between anxiety 
els of. cognitive 
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Experiment I 
METHOD 
Evaluation Instruments 


Four evaluative instruments were constructed to 
reflect four of Bloom's (1) classifications for cogni- 
tive behavior: Knowledge, Comprehension, Appli- 
cation, and Analysis, The content of the four multiple- 
choice instruments developed was taken from the 
physical and biological sciences. To ascertain the 
appropriateness of the test items (e.g. level of dif- 
ficulty, clarity of wording, and ability to discrimi- 
nate) a pilottry-out of seventy-eight of the *'initial- 
ly generated' items was conducted. From this it 
was possible to create helpful guidelines for further 
item writing and selecting. 


Four individuals with familiarity and experience 
in using Bloom's taxonomy, independently assigned 
each of the resultant test items to one of the four 
cognitive classifications being considered in this 
Study. Item classifications not agreed upon by at 
least three of the four persons were either rewrit- 
ten to become ‘‘acceptable” or were omitted, The 
final group of test items employed was selected 
through item analysis procedures. Split-half relia- 
bilitites of the four evaluation instruments, cor- 
rected with the Spearman-Brown Prophecy For- 
mula (12) were: Knowledge, . 81; Comprehension, 
+ 76; Application, . 85; and Analysis, .61, 


SAMPLE 


92 men) enrolled in Education or Western Civiliza- 
tion courses at the University of California, River- 
side, Junior, Senior, or fifth-year Standing is re- 
quired for admission to Education courses, Western 


all students, with the large majority of students com- 
pleting the requirement in either their freshman or 
sophomore year, Thus, the Sample was judged as 
generally representative of the population of students 

at the University of California, Riverside, 


Procedures 
All Ss were randomly assigned to one of four 


groups: Knowledge (K); Comprehension (C); Appli- 
cation (Ap); and Analysis (An), All Ss were admin- 


istered the Test Anxiety (TA), General Anxiety(GA), 


and Need for Achievement (NA) sections of the Auto- 
biographical Survey developed by Sarason (7), Test- 
retest reliabilities above .78 for TA and GA, and 
above . 70for NA have been reported (Sarason, 1958). 
Evidence of the equivalence of the groups was offered 
through analysis of variance for each of three inde- 
pendent variables: TA (F=0, 33; d, f. -3/233; N.S., 
p: .01), GA (Е=0. 51; d. f, =3/233; N.S., р< .01), 
and NA (F-1. 38; d. f. =3/ 233; N.S., p- .01) . 


One week after taking the tests described above, 
all Ss were given the science test appropriate to their 
group (i.e, K,C, Ap, or Ап). No time limits were 
imposed for any of the tests, 


RESULTS 


Correlational analysis was performed on the to- 
tal test scores for each group. Theassumption of 
linearity was confirmed by visual inspection of the 
graphed data, The correlation matrix for the group 
receiving the Knowledge science test (N=59) is re- 
ported in Table 1. Positive correlation coefficients 
departing Significantly from zero were between Gen- 
eral Anxiety and Test Anxiety (p< .01); and Test 
Anxiety and Need for Achievement (p<.05). The 
negative correlation between Test Anxiety and Know 
ledge was significant (p<.05). 


TABLE 1 


MATRIX OF INTERCORRELATIONS FOR KNOW - 
LEDGE GROUP 


————————————= 


Variable 1 2 3 4 
"———— 
l. Test Anxiety - 


2. General Anxiety .63** - 


3. Need for Achievement .28* .11 - 


4. Science Knowledge -.27* -.21* -.13 - 


————À—À € 


*p< .05 
** p< .01 


Results of analysis of the data from those subjects 
who took the Comprehension science test (N=58) are 
shown in Table 2, Significant positive correlations 
were found between TA and GA (p< .01); and GAand 
NA (p< .05). Significant negative correlations were 
between TA and Comprehension (p< . 01); and GA 
and Comprehension (p< .05). 


TABLE 2 


MATRIX OF INTERCORRELATIONS FOR COM- 
PREHENSION GROUP 


кеш—————————= 


Variable 1 2 3 4 
C E 
1. Test Anxiety = 

2. General Anxiety .63** = 

3. Need for Achievement .19 . 82* - 


4. Science Compre- 


hension -.43** . 298 1.97 - 
"pe 05 
рс „01 
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Table 3 is the correlation matrix of the scores 
obtained from the group which received the Applica- 
tion science test (N=62). Significant positive corre- 
lation coefficients were found between GA and TA 
(p< .01); NA and GA (p «.01); and МА and GA 
(p< .01). 


TABLE 3 


MATRIX OF INTERCORRELATIONS FOR APPLI- 
CATION GROUP 


_——ЄЄ————— 
Variable 1 2 3 4 


—————————————— 


l. Test Anxiety s 


2. General Anxiety — ,64** % 
3. Need for 
Achievement .60**  .51** = 


4. Science Appli- 
cation “17 -.03 


oe 


**5«.01 


The only significant correlation for the group 
which received the Analysis science test (N=55) was 
positive and between TA and GA ( p<.01). Table4 
reports the correlation matrix, 


TABLE 4 


MATRIX OF INTERCORRELATIONS FOR ANALY- 
SIS 


=——Є———————— 


Variable 1 2 3 4 
—————————— 
l. Test Anxiety ы 
2. General Anxiety .67** - 
3. Need for 

Achievement .21 .14 - 
4. Science Analysis -.02 .09 -.11 - 
p< .01 
DISC USSION 


The purpose of this investigation was to examine 
the interaction of various states of affective arousal, 
45 measured by GA, ТА and NA, and level of cogni- 
tive functioning (Knowledge, Comprehension, Appli- 
cation, and Analysis), Hence, the intercorrelations 
found between GA, TA and NA for all groups tested 
is of no particular interest; although one obvious con- 
clusion is they do not Seem to be measures of inde- 
pendent affective states. Although there appears to 
be a lack of consistency of correlations between TA 
and NA ( significantly different from zero for the 


Knowledge and Application groups); GA and NA (sig- 
nificantly different from zero for the Comprehension 
and Application groups), such is not the case, Inter- 
correlations between all groups were not significantly 
different (rom one another: TA and GA ( X^-, 163; 
d. f. -3; N.S., р<, 01); СА and NA (X2-6,85; d. f. = 
3; N.S., р< .01); TA and NA (X2-8, 83; d.f. 23; 
N.S., р<.01). 


The significant negative intercorrelations found 
between TA and Knowledge, TA and Comprehension, 
and GA and Comprehension indicate that states of 
anxiety are differentially related to levels of cogni- 
tive functioning, That is, higher levels of TA have 
a debilitating effect on the “lower” but not the 
*'higher'' levels of cognitive functioning. The signi- 
ficant negative correlation between GA and Compre- 
hension probably appeared because of the lack of in- 
dependence between GA and Th. 


The findings of Speilberger (10) which indicated 
that Ss with high and low anxiety function at the same 
level when unambiguous andclearly worded problems 
are presented provide a possible analogy to the re- 
Sults of this study, Although the protocols used were, 
hopefully, clear and unambiguous, the types of ques- 
tions asked and the information given for each of the 
four levels of Bloom's (1) taxonomy used provides 
us with the basis for our analysis. 


dent body from the upper 12 percent of high school 


viduals with high TA and GA, which resulted in their 


high and low anxiety and the type of Question, which 
implies levels of cognitive functioning required to 
give an answer » Played a crucial role intest 
performance, 


Experiment II 
Whether or not the relationships betwee 


METHOD 
Evaluation Instruments 
m struments: 


Thefour evaluative instrument; i 
s utiliz ^ 
ment I were renormed for the Ss of Experi 
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The following reliabilities (Kuder-Richardson, For- held for a younger, generally less pu e 
mula 20) were calculated: Knowledge, . 73; Compre- less test sophisticated population. Hence; АЕ snes 
hension, .73; Application, . 81; Analysis, . 72. ment II, a sample of junior and senior class hig! ae 
students of average ability froma middle to lower à 
dle SES area waschosen. Proceduresgenerally rep 3 
licated those employed in Experiment L The B 
dicated a significant negative correlation (p <. khe 
tween Test Anxiety and Comprehension. Thus d 
findings of Experiment I were partially replicated. 


Sample 


The sample consisted of 319 junior and senior class 
students fromthe high school of a unified school dis- 
trict. STEP-SCAT data indicated a sample which was 
academically average. The school serves a middle 
to lower middle SES community. 


FOOTNOTE 
ЖЕЕ i d 
Procedures 1. The research reported here was partially su eng 
i h nt, Univer: 
As in Experiment I, Ss were randomly assigned by о кене генш гаш, 
to one of four groups: Knowledge, Comprehension, or сашогиа, Ш 


Application, or Analysis, One week after taking the 
Test Anxiety part of Sarason's Autobio; raphicalSur- 
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ABSTRACT 


The Kounin (4) concept-switchingtask was administered to groups of institutionalized mentally retarded, 
non-institutionalized mentally retarded, lower-class normal, and middle-class normal children matched on 
MA, Theconcept switching task was administered under three different reward conditions: promise of reward, 
low tangible reward, and high tangible reward. Differences in performance between groups was tested by means 
of a 3 х 4 analysis of variance. F tests yielded no significant differences at the . 05 level. The study failed to 
confirm either Kounin's concept of rigidity or Zigler's deprivation motivational hypothesis. Possible reasons 


for lack of confirmation are discussed. 


EVER SINCE Lewin (5) postulated that the differ- 
ence between a feebleminded and normal child of 
the same degree of psychical differentiation (mental 
age) is inthe smaller capacity for dynamic rearrange- 
ment in the psychical systems of the feebleminded , 
the notion of rigidity has intrigued investigators. 
Kounin (4) offered the resultsof five experiments to 
demonstrate the tendency of the mentally retard- 
ed person to cling to afixed goalor habit thus making 
it difficult for him to evaluate new situations. 


Studies by Zigler and Unell (11), Zigler and 
deLabry (9), and Zigler, Hodgen, and Stevenson (10), 
however, suggest that incentive variables significant- 
ly affect experimental performance. Additionally, 
Heber (3) reported that an incentive which is highly 
preferred by the individual being tested seems to 
produce superior responses regardless of intelligence 
level, 


Investigations of the effects of institutionalization 
and social class оп various incentive responses have 
been limited, but provocative. Terrell, Durkin, and 
Wiesley (7) found that middle-class normal children 
Dom to desire to perform correctly for the sake of 
ae ing correct regardless of the incentive offered. 
thee Class individuals have tended to function at 
feat best levels when a tangible or material incen- 
i 1s offered as compared to an intangible incentive. 
е ышы to Zigler and deLabry (9), institutionalized 
aa ally retarted individuals and lower-class indi- 

iduals matched on mental age have performed com- 
Parably under the same incentive conditions. 


It is Zigler’s contention that institutionalized sub- 
jects tend to prolong adult interaction by continuing 
their tasks as long as possible. This is interpreted 
to reflect motivation rather than rigidity. In addition, 
Zigler (8), Backer (1), and Zigler, Hodgen, and 
Stevenson (10) found evidence that the institutionalized 
subjects tend to make few errors on their tasks. They 
consider this also due to a high motivation to secure 
adult approval. 


A study by Terrell (6), while it did not deal with 
retarded subjects, included a promise of reward. The 
promise of a reward, when compared with more tan- 
gible rewards, was less motivating for the subjects, 


Zigler (12) has proposed that the lack of psychic 
flexibility of the mentally retarded may not be due to 
rigidity as suggested by Lewin, but may rather be 
due to social deprivation (lack of adequate affectional 
adult contact). If this is correct, one would expect 
mental retardation, institutionalization, and social 
class to effect differential performance under different 
reward conditions when the performance task requires 
cognitive flexibility. 


This notion has been investigated i 
In addition to the original Study Мі oon 
concept-switching, two other studies have use 
same materials and tasks. Zigler and Unell (11) 
compared the performance of a group of instituti 
alized retardates with a group of normals usi a 
nonreinforcement condition and a reinforced ой 


)on 
d his 
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They found that both groups performed better under 
tangible reinforcement, but the normal subjects did 
no better than the retarded regardless of therein- 
forcement. This they interpret as supporting the 
motivational hypothesis. 


Zigler and deLabry used the concept-switching 
task of Kounin with an institutionalized mentally re- 
tarded group, a group of lower class normals, and 
а groupof middle class normals. They used two re- 
ward conditions: intangible (being correct) and tan- 
gible (choice of a toy). Their principle finding was 
thateachgroup performed best under optional reward 
conditions; intangible for the middle-class normals 
and tangible for the mentally retarded and the lower- 
class normals. This evidence tends to support the 
motivational hypothesis of Zigler. 


STATEMENT OF THE PROBLEM 


The purpose of the present study was to compare 
the performance of four groups of subjects of com- 
parable mentalages onthe concept- switching task used 
by Kounin (4) under three different reward conditions, 
The groups were: 


1) Institutionalized educable mentally retarded, 

2) Non-institutionalized educable mentally 
retarded, 

3) Middle-class normals. 

4) Lower-class normals, 


The reward conditions were: 
1) Promise of a reward. 
2) Low tangible reward. 
3) High tangible reward, 
METHOD 
The subjects for the present study included thirty 


non-institutionalized mentally retarded children se- 
lected from special education classes at the Colorado 


TABLE 1 


MEAN CHRONOLOGICAL AGES (IN MONTHS), 
AND IQ’S OF ALL EXPERIMENTAL GROUPS 


Groups Inst. Non-Inst, Lower- Middle- 


Class Class 
E 
MR EMR Normal Normal 


Promise of CA 162.6 135.2 95.2 88.4 


Reward MA 90.9 84.2 98.7 93.5 
IQ 65.2 68.5 96.1 102.9 
Low CA 155,7 148.1 103.2 91.9 


Tangible МА 85.9 85.9 95.3 91.1 
Reward IQ 63.6 66.4 95.8 99.3 


High | СА 161.6 145.0 91.2 93.9 
Tangible MA 88.7 87.9 97.6 98.6 
Reward IQ 63.4 67.0 105.2 102.6 


NOTE: CA--Chronological Age, MA--Mental Age, 
IQ --Intelligence Quotient 


State College Laboratory School and the Waco [ш> 
pendent School District, Waco, Texas. Thirty ins d 
tutionalized mentally retarded subjects were selecte 
from the Mexia State School in Mexia, Texas. A 
total of sixty normal subjects were selected ов. 
elementary schools in the Waco Independent Schoo 
District. Thirty normal subjects were chosen from 
middle-class socio-economic backgrounds as deter 
minded by a modified versionof the Warner's Tees. 
of Social Characteristics (2), and thirty normal A 
jects were from lower-class socio-economic bac 
grounds as determined by the same Index. 


All subjects were administered the Peabody P 
ture Vocabulary Test to determine mental age an 
IQ scores. The characteristics of the groups are 
given in Table 1, 


Kounin's concept- switching task (4) was s 
tered to determine the subjects’ abilities to SV к 
concepts under three rewardconditions. Тһе гема à 
conditions were: 1) a hightangible reward of rt n 
2) a low tangible reward of aslip of paper with whi а 
to redeem an unknown reward later, 3) anda prom 
ise of a reward. The description of the concept- 
Switching tasks and the procedures following the 
administration are as follows: Task I, a color 
Grouping, and Task П, a form grouping, were pre- 
experimental. These tasks gave the su bjects 
experience with two principles of classification and 
indicated whether or not they could make the appro- 
priateformandcolor discriminations. TaskIII w as 
the experimental task involving either a color or a 
form grouping. 


Task I.--Twenty-five colored rectangular cards, 
measuring 3.5x 2.5 inches, were presented to each 
subject who was asked to classify them into groupings. 
There were five eachof the following colors: orange, 
yellow, green, blue, and red, The instructions for 


this and subsequent tasks were similartothose used 
by Kounin: 


Here are some things all mixed up. 
Some of these are the same and belong 
together. Here is a box with five sec- 
tions. Suppose you find some that are 
the same and belong together and you 
Put them in one of the sections. Then 
find some others that are the same and 
Put them in another section, and keep 


going until you have all those that are 
the same together, 


е used. 


Task П. -- Twenty-five white cards wer hapes: 


There were five cards of each of the following ° inted 
triangles, Squares, circles, crosses, and р who 
Stars, Th | 


These were presented to each subjez: ings- 
was again required to classify them into five rane 


Task Ш, -- 
colors were t 
whose forms 


Twenty-five cards were used унон, 
һе same as those used in Task T cux 
Were the same as those used in bes 
П. Again, there were tive of each type of form, wi 

each of the Sameform painted a different color, 1. е., 
ап orange, yellow, green, blue, and red triangle, 


ete. Eachcolor was represented in each shape once 
This task had two parts: 


Part 1. --Тһе twenty-five cards were presented to 
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each subject, and he was asked to classify them into 
five groupings. 


Part 2. -- After the subject had classified the cards 
correctly, on the basis of either color or form, the 
experimenter reshuffledthe cards, returned them to 
the subject, and asked him to put them together some 
other way. 


PROCEDURE 


Each subject was administered the experimental 
taskindividually. Atnotime wasany form of verbal 
praise offered except in relation to the reward the 
subject was attemptingtoearn. The subject was told: 


We're going to play some games today 
and give you achance to win some prizes if 
you want to. Do you want to? You don't 
have to if you don't want to. Are you sure 
you want to? 


The subject was initially given one of the pre- 
experimental tasks, either Task I or Task II. After 
completing this task, he was given the other pre- 
experimental task, One-half of the subjects in each 
group was given Task I followed by Task II, with the 
other half being given Task II followed by TaskI. The 
subjects were given Part 1 of Task III, followed by 
Part 2 of Task III. 


In the high tangible condition (choice of several 
5 to 10 cent items such as gum, candy, andCracker 
Jacks), when the subject was seated, he was told: 


We're going to play some games in which 
you can win a prize. Every time you get a 
game right, you сап choose any one of these 
prizes. (The subject was shown the items 
mentioned above which were placed on the 
table. ) 


Upon successfully completing Task I, Task II, and 
Part 1 of Task III, the subject was allowed to take 
his choice of the items on the table. Prior to Part 
2 of Task III, which required the subject to switch 
concepts, the subject was told: 


See if you can win a prize by put- 
ting them together some other way. 


If he did not switch on the first trial, he was then 
told: 


You didn’t win a prize that time. Seeif 
you can put them together some other way. 


If the subject still did not change his method of 
£rouping, he was instructed as follows: 


No, you can win a prize by finding some 
other way that they may belong together. 


єн latter instructions were continued untilthe 
Subject had a maximum of seven total trials or a 
correct grouping, either of which terminated the 
experiment for the subject. Ifthe subject hesitated 


1 continuing during any of the seven trials, he was 


Sure you can do й. Remember, you 
can win a prize if you can find some other 
way to do it. 


If the subject did not switch concepts by the seventh 
trial, he was terminated, and was automatically given 
a score of eight. 


In the low tangible condition, the subjects were 
given the same instructions, but instead of showing 
them the items onthe table, they were shown a stack 
of slips of paper, marked “Опе prize (1)’’ lying on 
the table. If a subject was unable to read the slip of 
paper, it was read to him. Ashe successfully com- 
pleted each part, he was given a slip of paper. The 
words ‘‘slip of paper” were substitutedfor “ргіле” 
throughout the experiment using the low tangible con- 
dition, 


In the promise of reward condition, the subject 
was not shown anything. The initial instructions were: 


We're going to play some games. For 
every game you get right, you will receive 
a prize after everyone else has hadachance 
to play the games. We will play four games, 
so if you get all four right, you will receive 
four prizes, 


The remainder of the instructions were the same 
as for the high tangible condition, with the comment 
after each successful part: 


That was right. After everyone else has 
achance to play the game, you will get a prize 
for that game. 


Since the promise of reward condition groups were 
tested first, these subjects had no knowledge of the 
nature of the prize for which they were actually work- 
ing. The low tangible group was tested afterthe 
promise condition was completed, so that this group 
had only the slip of paper which they were told they 
would be able to redeem for a prize, the nature of 
which was unknown to them, 


Because of the need for sequencing the reward 
conditions from promise to low tangible to high tan- 
gible, subjects were assigned to conditions on the 
basis of the order in which they were identified: the 
first ten S's of group 1 to condition 1, and the next 
10 S's of group 1 to condition 2, etc. 


FINDINGS AND DISCUSSION 


The groups were tested for differences by means 
of a 3x 4factorialanalysis, In testing for an inter- 
actioncomponent, an F test was utilized. The F test 
resulted ina value of 1.146 which was not significant 
at the . 05 level of confidence, ` 


Ап Е test was utilized to determine si, nifie: 
of main effects involving the four groupe This re- 
sulted in a value of .045 which was not Significant at 
the . 05 level of confidence. 


An F test was utilized to determine significances 
of main effects involving the three reward conditions, 
This resulted in avalue of 2.122 which was not sig- 
nificant at the . 05 level of confidence. 


24 THE JOURNAL OF EXPERIMENTAL EDUCATION 


Of the 120 subjects inthe experiment, eight failed 
to complete the task of concept switch. Multiple re- 
gression analysis was therefore used to examine for 
possibledifferencesbetween subjects in the four 
groups and under the three reward conditions using 
only those 112 subjects who were successful in ac- 
complishing the switch. Using number of trials to 
completion as the criterion, cell membership in the 
3 x 4 design and IQ as predictor variables yielded a 
multiple R2 оѓ. 075 and a non-significant F of . 74. 


Table 2 is a summary table presenting the statis- 
ticaltreatment of the number of trials to completion 


on the concept Switching task for all subjects in the 
experiment, 


TABLE 2 
ANALYSIS OF VARIANCE FOR TRIAL TO 
COMPLETION 


Source Sum of Squares DF M-Square F 


Columns 0. 3166 2 0.1583 0.045 
(Rewards) 

Rows 22, 2916 3 7.4305 2.122 
(Groups) 

Interaction 24, 0833 6 4.0138 1.146 


Within Groups 378. 1000 108 3. 5009 


Total 424, 7916 119 
——— d. ———————— 


DISCUSSION 


The data from the present study failed to support 
either Kounin's theory of psychic rigidity of Zigler's 
differential motivational hypothesis concerning in- 
Stitutional and social deprivation. This was true 
whether all the subjects were included in the analy- 
Sisor whether only those who Successfully performed 
the concept Switching task were compared. Just why 
this empirical test failed to substantiate either 


Kounin's theory or Zigler's hypothesis is not clear, 
Several possibilities can be considered, 


First, the subjects employed in the present study 
differed considerably from those employed by other 
investigators. The Subjects used in this study had 


a half older than those 


account for the findings, 


Second, it is possible 
red in school settings wh 
for some time, that a 


li 
paper believed to be eh 


redeemable for a reward and 


an actual immediate prize were perceived by ee 
subjects as comparable motivating factors. Tee 
the school experiences of the subjects may have ed 
such that they learned to believe that school associa 
adults keep their promises. 


The findings of the study and the discussion sug- 
gest: 


1. Kounin's theory of rigidity and Zigler's, ai 
ferential motivational hypothesis are no 
able. 


i le 
2. Boththetheory andthe hypothesis are tend 
but are modified as a function of age а 
mental age. 


; able 

3. Both the theory and the hypothesis аге tenome 
but are modified as a function of expe 
and/or training. 


The needfor further research is self-evident. = 
though studies utilizing designs similar to the pre the 
study could use populations similar to those of ffi- 
earlier studies, it would seem somewhat more € 5s 
cient to employ a complex design which might allo 
for the identification of possible interaction effects. 


In any case, the implications for classroom pre- 
cedures are muchtoo important to be ignored. If in- 
deed tangible, low tangible and promises of reward 
are equally motivating, classroom reinforcement 
can be greatly simplified. Similarly, if response to 
various types of reinforcement can be altered by ex- 
perience or is altered by maturation, this may have 
an effect on classroom procedure, 


Conversely, should future research sustain the 
Kounin theory or the Zigler hypothesis, this may gen- 
erate a reexamination of this important issue. 
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ABSTRACT 


PUPIL SCORES on objective tests are often used 
as a basis for judging the relative efficacy of educa- 
tional innovations or ‘treatments,’ such as different 
curricula, instructional devices, or teaching meth- 
ods. The scores are usually analyzed in factorial 
designs at various levels of complexity. However 


(9:253) in this Journal through the following exam- 
ple: 


Suppose a fifty item test is given to two 
Classes... of arithmetic students who were 


would be Sustained, 


In the above example, the methods under com- 


in the ‘treatments’ as another factor—it may also c ontain 
antiated by the results from a four factor design, containing 


parison would prove to be equally effective hue d 
judged on the basis of total scores, On the [c 

of the items in the first half of the test, d d 
the first method would prove to be superior, 1A 
reverse being the inference from the items Mt eS 
Second half of the test, The contradiction ar reat 
from the presence of interaction between the lves 
ments’ and the (items belonging to the) two ha that 
of the test. Moonan recommended, tere 
“‘we need to use an examination with items c factors 
are affected by, but do not interact with, the 

of experiment, " 


at 

Since a researcher cannot ensure a priori ith 
the test items he plans to use will not аА е 
the ‘factors or experiment, ' he has to ug the test 
Problem a posterior; After having selecte and 
on the basis of other relevant considerations P 
using it, he can analyze the data to find out tments’ 
the items of the test interacted with the ыра esent. 
and with other “factors of experiment,’’ if P icd 
Such analyses involve the application of pisc 
measurements designs, such that every item tor 
as a criterion and Constitutes a level of the fac ou 
containing repeateq measures, Such a design Wi to 
ast one more factor—that pertaining 


e 
the ‘treatments’ being compared, It may also inclu 


X 
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additional factors or variables which the researcher 
may like to examine simultaneously. 


The various factors of a repeated measure- 
ments design can be visualized through a two-way 
layout in which the column headings represent test 
items furnishing repeated measures and the row 
headings stand for the various groups of individuals 
answering the items. 


The simplest design would contain only two 
factors, one having repeated measures and the other 
‘treatments.’ In such a design, the subjects are 
classified only in terms of the ‘treatments’ adminis- 
tered to them. In the three factor design with repe- 
titions on one factor, the subjects are also classified 
on the basis of an additional variable, for example, 
sex. In the four factor design of this type, classifi- 
cation is based upon yet another variable, such as 
the socio-economic status or ability levels of the 
subjects, and so on. Similarly, instead of a single 
factor containing repetitions, one canuse two or 
more such factors. 


Irrespective of the number of factors included 
in a repeated measurements design, the analysis of 
data contains two parts: 
(a) a ‘between subjects’ or ‘between subjects 
within groups’ part in which the total scores 
constitute the criterion. As such, the hypoth- 
eses examinable in this part of the analysis are 
called here ‘score’ hypotheses. Most research- 
ers limit their analysis to testing such hypoth- 
eses only, 


(b) a ‘within subjects’ part in which each test 
item furnishes a criterion. Hence the hypoth- 
eses testable under this part are termed here 
‘item’ hypotheses, 


The present paper aims at showing the limita- 
tions inherent in confining attention to ‘score’ hypoth- 
eses and at exemplifying the contribution which an 
examination of ‘item’ hypotheses can make to the 
evaluation of ‘treatments.’ 


METHOD 


The design of the experiment used in this re- 
search had four factors, of which three were classi- 
ficatory in nature, Factor A consisted of fifteen 
mathematics teachers in the U, S. Midwest who 
voluntarily participated in the study. Factor B rep- 
resented the students’ sex. Factor C denoted two 
types of textbooks--‘SMSG’ and ‘conventional,’ Each 
teacher had taught concurrently two sections of a 
к grade, one using an ‘SMSG’ text developed by 
he School Mathematics Study Group (1), and the 


e Section using any of the various ‘conventional’ 


The пеге меге 513 subjects— 232 girls and 281 boys. 

text was used by 255 of them and the ‘con- 

€xts by 258. Two-thirds of the teachers 
Оту assigned the studentsto the two types 
i a peu of the subjects' responses to 

ematics na Tests of Educational Progress, Math- 
' “orm 2A (STEP Math 2A) published by the 
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Cooperative Test Division of the Educational Test- 
ing Service. The test was answered by the subjects 
on entering the study and again after a calendar year. 


An assumption under which repeated measure- 
ments designs can be applied to item responses is 
that the variance-covariance matrices of the re- 
sponses for the various populations are equal and 
symmetrical (2,3,4, 5,7, 8,10, П). To ensure that 
the items furnishing the repeated measurements con- 
formed as much as possible to this condition, the 
pretest responses were factor and cluster analyzed. 
The results (6) led to a split of the STEP-Math 2A 
into three parts containing 16,19, and 15 items. The 
first part was highly homogeneous (KR-20 of . 85). 
As such, its items were used as constituting the var- 
ious levels of factor D. Since all the levels of each 
of the four factors were included in the experiment, 
the analyses were done in terms of a fixed model, 
The design was applied separately to the pre and the 
post responses. 


RESULTS 


The results of the analyses are given in Table 
1. Fifteen rows of Table 1 are numbered, each rep- 
resenting a testable hypothesis. Instead of describ- 
ing the results related to all the fifteen, an attempt 
is made here to deal with only those hypotheses in 
which the test items showed interactions with the 
classificatory factors and their implications for the 
‘score’ hypotheses. They are: 


(a) Item-teacher interaction was found both 
in the pretest and the posttest data (row no. 9). This 
indicates that when the experiment began, the stu- 
dents taught by the various teachers did not have uni- 
form background in relation to the subject-matter 
areas covered by the test items and that the situation 
had not changed at the end of the experiment which 
was after a year. The finding has direct implica- 
tion for the *score' hypothesis on differences be- 
tween the teachers (row no. 1). Because of the pre- 
sence of item-teacher interaction, ‘between teachers' 
differences should be approached through an examina- 
tion of ‘simple’ main effects rather than that of the 
*overall' main effect, 


(b) There was item-sex interaction at the pre- 
test stage (row no. 10). Further data analysis seems 
to show that the items were classifiable in three 
subgroups such that on the first group the boys did 
better, on the second equally well, and on the third 
worse,when compared to the girls. The observed 
absence of a sex difference (row no. 2) is thus mis- 
leading and reminds one of Moonan's example cited 
earlier. 


Here again, an examination of ‘simple’ main 
effects is necessary. 


(c) Item-text interaction was present on the 
pretest occasion (row no. ll) showing that when en- 
tering the investigation, the students of the SMSG 
text had better preparation than their counterparts 
on some items and that the reverse was probably 
true of other items. Item-text interaction may have 
arisen from the failure of about a third of the teach- 
ers to allocate the students randomly to the two types 
of texts. Its presence, however, does imply that 
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TABLE 1 


PRE AND POST ANOVA TABLE FOR FOUR FACTOR DESIGN: REPETITIONS ON ONE FACTOR 


Pretest Posttest 
SV df MS F MS F 
(1) A:Trs. 14 2.216 27. 56** 3. 323 87, 2707 
(2) B:Sex 1 .170 2.11 .011 .00 
(3) C:Texts 1 . 002 .03 . 002 .02 
(4) AB:Tr.-Sex 14 .050 .62 . 045 #91 
(5) AC:Tr. -Text 14 .119 1.49 ‚191 14 
(6) BC:Sex-Text 1 .012 .15 . 009 #10 
(7) ABC:Tr.-Sex-Text 14 .052 ‚65 ‚042 EE 
Sub. w. groups 453 . 0805 . 0894 
(8) D:Items 15 8. 410 55. 40** 10. 119 11.56 ** 
(9) AD:Tr.-Item 210 .361 2.38** .317 2, 24** 
(10) BD:Sex-Item 15 . 270 1. 78* .179 3.29 
(11) CD:Text-Item 15 . 266 1. 75* .115 .81 
(12) ABD:Tr.-Sex-Item ` 210 .125 .82 .159 1.12 
(13) ACD:Tr.-Text-Item 210 .159 1.05 .126 .89 
(14) BCD:Sex-Text-Item 15 .136 .89 .030 .21 
(18) ABCD:Tr. -Sex-Text-Item 210 .172 1.13 Ji 278 
DxSub. w. groups 6795 


**Significant аё. 01. 


the acceptance of the null hypothesis related to the 
differences between the texts (row no. 3) is not jus- 
tifiable, 


Since item-text interaction was not found in 
the posttest data, one could also conclude that the 
texts had something to do with eliminating the item- 
text interaction present in the pretest data. In this 
sense, then, the texts did differ, in spite of the con- 
clusion to the contrary from row no. 3 of Table 1. 


DISCUSSION 


In all the researches in which performance on 
an objective achievement test furnishes the criterion 
measures likely to lead to certain conclusions, the 
researcher has two major choices: (i) He can use 
all the items of the test as a group and derive a 
single score for each Subject. This is the most 
common practice. (ii) He can use each item inde- 


pendently and derive as many 'scores' or measures 
for each individual as there are items. (ii) is ° addi- 
sentially an expension of (i) in the sense that E $ 
tional factor is added to the experimental des Е 
implicit in (i) and in the process, (i) is transfor 

into a repeated measurements design. 


The extra computational labor involved E He 
application of a repeated measurements des f the 
only a negligible fraction of the total cost Е rich 
Tesearch in the age or computers and can P ted by 
dividends. That this is so is clearly suppor lusions 
the evidence presented here. From the concu ms 
cited above, it is clear that whenever test ite 5 
Show interaction With one or more of the inet te il 
of experiment" (9), manyof the conclusions ee 
rived through the use of alternative (i) would i ы 
to be re-examined through an analysis of ‘simp d 
main effects. Alternative (ii) is, therefore, rec 
ommended for comparing ‘treatments. ’ 


NW 
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ABSTRACT 


The increasing size of most universities in the United States 
largeness by creating many small units within the larger one. 
evaluate the effectiveness of a grouping pattern in which fresh 
in such a way as to have at least three sections (courses) in c 


THE NEED for some new ideas concerning student 
group identification, such as cluster grouping, re- 
sults at least in large measure fromthe tremendous 
growth of student enrollment in A merican institutions 
of higher education in the last quarter of acentury 
(11. This steadily increasing enrollment in many 
institutions of higher education has been aconcernto 
administrators who fear that increase in the size of 
institutions will be detrimental to the intellectual 
growth and personal development of the students (9). 
One of the proposed solutions to the problem has been 
clustering students into various types of relatively 
Small groups such as groups based on common resi- 
dence, common course Sections, or both (1, 5, 11). 
Some of the objectives of these types of groupings 
havebeen: 1) better andfaster adjustment to college 
by new students, 2) lowering the dropout rate, 
3) closer professor-student relationships, and 4) 
more meaningful coursework as a result of coordin- 
ating assignments, for example, an English theme 
might be written on the history topic for that day. 


History of Grouping Students 


Features of cluster grouping have had a long his- 
tory in the English universities of Oxford and 
mbridge (11). The Claremont Colleges were among 
the first in the United States to attempt to have stu- 
dents share facilities yet retain their identity with a 
The University of the Pacific first 
applied the term, *'cluster college," to a group of 
tutions on a single campus. The 
planned and built around the clus- 
Е S the Santa Cruz campus of The 
University of California (11). The cluster college 
С vides that a given number of students 
live and usually receive instruction in а common 
8roupof facilities, But since many institutions have 


at times has ledto efforts to counteract 
The following study reports on a pilot study to 

man students had their class schedules arranged 

ommon with others in their respective group. 


a large number of commuting students or have facil- 
ities not designed to accommodate small living- 
learning units, interest has at times developed in 
Structuring cluster groups (as opposed to cluster col- 
leges) which would still serve as primary groups 
within the context of the large university. 


Related Research 


Several universities throughout the United States 
currently are exploring the idea of cluster grouping 
or have already involved themselves in such a pro" 
gram (2, 3, 5, 9, 10, 11). However, the only objec 
tive evaluation of cluster grouping in the eru 
search appears to be one undertaken in The am a 
State University (9). Researchers in Florida ata x 
report measures of extra curricular participation ds 
informal social participation, time devoted to stu T 
ing, perceived stress, and grade- point average rye 
no systematic differences between groups. While a 
experimental groups had a lower attrition aont 
Stakenas (9) raised the question of a possible sa le 
pling error. Generalsatistactiondata, for PUMA 
in terms of satisfaction with courses, аатор 
cial activities, new friendships, were higher eat 
clustered students (p< - 025) at the end oF the icant 
term, but the difference did not remain sign rticu- 
(p=. 10) at the close of the school year- of Lo res- 
lar interest was the decrease in satisfaction сана 
Sed by the living-learning clusters and the = housed 
in satisfaction expressed by both the randomly es of 
clusters and the nonclustered groups. Медон. гіп 
Berceived peer pressure were found to begreate 
the clustered groups, 


Problem 


" ity of 
Recently the administration of The University o. 
Tennessee (0. T.) became actively interested in 
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idea of grouping students. Student enrollment had 
been increasing rapidly and the dropout rate was of 
increasing concern. As a consequence a pilot study 
was developed forthe FallQuarter, 1967, to investi- 
gate the effects of cluster grouping. 


Definition of Cluster Grouping 


First-quarter freshmen enrolled in the College 
of Liberal Arts, having a common residence, and 
having three common course sections made up the 
cluster groups. First-quarter freshmen were chosen 
because one of the proposed and expected effects of 
cluster grouping was a better and faster adjustment 
to college. Using new freshmen eliminated the con- 
tamination of prior college experience. Students in 
the College of Liberal Arts, which has diverse course 
offerings even at the first-year level, comprised the 
experimental and control groups. Other colleges 
tend to have a natural course grouping for their stu- 
dents by virtue of the requirements within the col- 
leges. A newly opened residence hall, with separate 
wings for men and women in a new dormitory com- 
plex, served as the place of residence for three rea- 
sons: better environmental control existed due to a 
board-and-room combination; it was assumed that 
students living in this area would be from a somewhat 
similar socio-economic background; a residential 
control group could be selected from two similar dor- 
mitories in the complex. 


Students and their advisers selected the courses 
to betaken. Followingthis selection, schedules were 
developed so that each experimental group had at 
least three common courses with identical section 
assignments. 


METHODOLOGY 
Experimental Groups 


Five experimental groups were formed based on 
three common courses, Freshman courses in Eng- 
lish and History served as two of the three courses 
in each of the five groups. The third course for 
Group 1 was French, for Group 2 Chemistry, for 
Group 3 Zoology, for Group 4 elementary Spanish, 
and for Group 5 intermediate Spanish. Since the 
class size for English is restricted, a maximum 
size limit of 27 was set for each experimentalgroup. 


Finalgroup membership. In order tobe qualified 


for group membership, as previously stated, status, 
residence, and membership in three experimental 
sections had to be met by each student to be consi- 
dered a member of the cluster group, While very 
few changed residence plans before Fall Quarter, 
many of the experimental members had to be drop- 
Ped from the groups because of requested section 
changes or dropping one or more of the three sche- 
шей cluster classes. No attempt could be made to 
onc ee students to stay with their group, because 
Sith M. variables to be studied was the rapidity 
оеган ich they became aware of their group mem- 
P. In addition, final group membership de- 


ni 
Study d nOn the returnof a questionnaire used in the 


Control Groups 
ps 
Residence control (RC). Freshmen liberal arts 


students from two other residence halls in the com- 
plex composed the RC group. In order to keep the 
proportion of men and women equivalent to the experi- 
mental groups, every fifth woman and every third 

man was selected from an alphabetized listing of 

qualified subjects. Again, to remain a member of 
this group, a questionnaire had to be returned. 


English control (EC). In order to consider the 


difference between student-instructor relationships 
in a cluster grouping study, it appeared of interest 
to introduce another factor. Of the fiveinstructors 
inthe experimental English sections, three had other 
freshman English sections. These sections were 
taught by the teachers who had experimentalgroups 
1, 3, and 5. Reference will be made hereafter to 
ЕСІ, EC3, and EC5 while the experimental English 
groups will be indentified by EE1, EE2, ЕЕЗ, EE4, 
and ЕЕ5. 


Other Students Surveyed 


Additional information was obtained by means of 
the questionnaires from those students scheduled in 
the EE classes but not members of thecluster groups. 
In many casesthese wereformer members who were 
dropped because of not fulfilling all the final require- 
ments. Some, however, were never consideredífor 
membership. These groups will be referred to as 
“‘outsiders’’ and will be designated as OE1, OE2, 
OE3, OE4, and OE5. 


Instrument Used 


An instrument was designed especially for this 
study to attempt an assessment of each student's con- 
notative meaning of the concepts, ‘Му University,’ 
“Му English Class, ” and ** Myself," as measured 
by a semantic differential technique (6). Adjective 
pairs from those used by Pervin (7) were placed on 
a 7-point scale to compare the concepts, “Му Uni- 
versity” and '' Myself," as well as “Му English 
Class" and ** Myself, ” 


This instrument also collected such data as sex 
age, size of high school graduating class, geographi- 
cal location of high School, fraternity or Sorority af- 
filiation, and whether The University of Tennessee 
had been the student's first choice in applying to col- 
lege. Students were asked about their awareness of 
being a member of a clustered group, and, if so 
when they became conscious of such clustering. The 
number of courses taken during the quarter was re- 
quested as well as the number of visits home. The 
respondents were asked to express in percentages 
the amount of time spent in Studying, recreation 
socializing, working, and other. A final question 
asked them to check the one group in which most of 
their friends were located with the choices bein, 
classmates, dormmates, fraternity-sorority NER 
school friends also enrolled at The University E 
Tennessee, non-university people, and other, 


The questionnaires were mailed to the members 
on November 17, 1967. Follow-up letters were sent 
a week later to those not having yet responded, Ad- 
ditionalquestionnaires were mailed to those request- 
ing them after having received the follow-up letter 
and not able to find the original, 
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Analysis of Data 


Semantic differential scales. The seven points 
on each scale were assigned the sequential numerical 
value 1 through 7. Instructions were to circle one 
number for each adjective pair or scale. The num- 
ber circled was the score assigned to that scale. 


Conceptcomparisonscores. A distance analysis, 
as suggested by Osgood (4) was used to study the re- 
lationoftwoconcepts. If they were alike in meaning 
for the individual, the distance in semantic space 
would be small and vice versa. Pervin (7) found 
Similarity in meaning related to satisfaction. 


The difference between the two scores on the same 
Scale for two concepts for one individual was desig- 
nated as “d,” The linear distance, over all scales, 
between any twoconcepts, the “D” statistic (6), was 


found for each individual using theformula: р =V zd? Y 


For each individual a Di Score for the concept com- 


parison, ‘Му University” and “Myself, ” and a sec- 
ond score, Do, representing the comparisonbetween 
“Му English Class” and “Myself” were calculated, 
Group comparison scores, In order to determine 
the connotative distance betweenthe two concepts as 
expressed by the group, D scores were averaged 
(D) across individuals {ог each group. Then t-tests 
were used to analyze the D scores for the experiment- 
alandcontrolgroups. Grade pointaverages (GPA) for 


TABLE 1 


Fall and Winter Quarters were also used to compare 
the experimental and control groups, again using 
t-tests. 


Ordered individual scores. The D, scores of all 


experimental and control members were ordered 
from the smallest or lowest (a similar meaning for 
the concepts, ‘Му University” and “Myself ) tothe 
highest and total group was divided into thirds, termed 
hereafter the bottom (lowest) third, middle third, 
and highest third. Chi-square values were calcula- 
tedto determine if there were any differences in per- 
Sonal data characteristics for the lower or higher 
third. Cells containinglessthanfive members were 


collapsed before the chi-square values were deter- 
mined, 


RESULTS 
Group Size 


Froman original N of 107, fifty-five of the experi- 
mental group were qualified for cluster group mem- 
bership. Questionnaires were returned by 91 percent 
of those qualified, leaving a total of fifty experimental 
group members. These were divided into their re- 
spective groups as follows: EE1--8, EE2--15, 
EE3--13, EE4--7, EE5--7, 


Of the seventy-one students selected for member- 
ship in the RC group, 73 percent or fifty-two students 
returned the questionnaire. Eighty-one percent 
(М = 22) of the females and 68 percent (N= 30) of the 
males responded, 


COMPARISON OF GROUPS USING SEMANTIC DIFFERENTIAL DATA 


Groups D, SD df t р 
Residential Control 
EE1, EE2, EE3, EE4, ЕЕ5 (N-50) 8. 5332 2. 4456 
100 0. 0080 nu 
RC(N-52) 8. 5294 2.3170 
Groups b» SD df t B 
English Control 
EE1 (N -8) 5. 2362 1.1176 п. S. 
-2, 9867 Р 
EC1 (N- 17) 4. 0029 0. 8871 М 
ЕЕЗ (N = 13) 6. 3562 2.3596 п. 5. 
-0. 9133 à 
EC3 (N = 16) 5. 5644 2.2909 s 
EE5 (N - 7) 3. 7000 1.3039 s 
EC5 (N = 18) ш ны ` 
4. 7917 1.7111 


«у 
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Questionnaires were returned by 69 percent of 
the EC group members. Thefinalgroup sizes were 
ЕС1--17, EC3--16, and EC5--18. 


The number of non-cluster members or''outsiders" 
responding to the questionnaire was: ОЕ1-- 14, OE2-- 
4, OE3--7, OE4--13, OE5--12. 


Group Concept Comparison Scores 


No significant difference (t, р>. 05) was found 
between the experimental group and the residential 
group using the semantic differential data (Table 1). 
For the ЕЕЗ and EC3 as well as the EE5 and EC5 
groups no differences were found. Forthe expected 
direction (one-tailed t-test) no significant difference 
(p >. 05) was found between the EE1 and EC1 groups. 
However, the absolute magnitude was such that had 
atwo-tailed t-test been indicated, the control group 
would have had significantly (p <. 01 ) closer concept 
meanings between “Му English Class" and “Myself” 
(Table 1). 


Using the р, Score as the basis of comparison, 


no difference was found between the clustered and 
nonclustered members in the EE sections (Table 2). 


GPA Comparison 


In comparing the experimentaland control groups 
on Fall and Winter GPA's, theonly significantdiffer- 


TABLE 2 


COMPARISON OF SEMANTIC DIFFERENTIAL DATA 
FOR CLUSTER AND NONCLUSTER MEMBERS IN AN 
ENGLISH CLASSROOM 


Groups р, SD df t p 


EE1 (N = 8) 5.2362 1.1176 


20 0.3916 n.s. 


OE1(N=14) 5.6300 2.6918 
ЕЕ? (N =15) 4.3533 1.9019 
17 -0, 2561 m.s. 
OE2(N = 4) 4.0900 1.4256 
ЕЕЗ (№ -13) 6.3562 2.3596 
18 0.0311 ns. 
ОЕЗ (ү = т) 6,3200 2.6947 
EE4(N.7) 55986 1.7475 
о 18 -0. 7889 n.s. 
E4 (N = 13) 4.7562 2,4641 
EES (№ -7) з. т000 1,3039 
17 1.2580 n.s. 
OE5 (N = 19) 4,7175 1.8821 


ence (p <. 05) was again a reversed difference with 
EC1 having a higher GPA than EE1 in the Fall Quar- 
ter. This did not hold into the Winter Quarter, how- 
ever (Table 3). 


Personal Information Data Characteristics 


Of the fifteen personal information data variables 
considered, only two characteristics were found to 
have significance (Table 4). Sex was significant 
(x2, р <. 05) witha greater percent of men in the 
bottomthirdof the ordered Di Scores (Table 5). The 


amount of time spent studying was the second signif- 
icant variable (x2, p <. 025). Percentages for these 
data are reported in Table 6. 


DISCUSSION AND SUMMARY 


If the cluster grouping of students had given its 
members an identity group, an attitude of greater 
warmth toward or satisfaction with the university 
would have been expected, as expressed by a differ- 
ence between the EE groups and the RC groupon 
D; scores. A closer student- instructor relationship 
as a result of cluster group membership would have 
been reflected by the EE groups having a closer mean- 
ing for the two concepts, “Му English Class" and 
** Myself," than the comparable EC groups. 


These differences did not emerge. Therefore, we 
must assume that cluster grouping as measured in 
this study proved ineffective, However, it should be 
pointed out that this experimentalsurvey had certain 
weaknesses in the design: 1) A disclosure tothe ex- 
perimental group members of their participation in 
a cluster grouping study was made after the question- 
naires were mailed but before all were returned, 
Such a disclosure should have inc reasedthe probabili- 
ty of getting significant results as it has been found 
(8) that increased attention to а group subsequently 
results in increased satisfaction for the group. 2) 
The liberal drop-add and section-change policy of the 
university allowed the cluster groups to decrease to 
Such size that they were the minority group even in 
the small English classes, thus destroying some of 
the possible effects of the treatment. 3) Three of the 
courses had extremely high drop-out rates and 
should not have been included in the group course 
clusters, 4) The control groups were not matched 
for these troublesome courses, 


The only two significant findings for group com- 
parison were reversed withthe control group exceed- 
ing the experimentalgroup. These involved the Same 
groups, EClandEE1. Since the differences involved 
GPA as well as satisfaction, assuming satisfaction 
to be measured by similar concept meanings, an un- 
measured variable, intelligence, may be interacting. 
Perhaps the more intelligent students adjust faster 
and better to the college environment, 
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TABLE 3 


GPA COMPARISONS FOR EXPERIMENTAL AND CONTROL GROUPS FALL AND WINTER QUARTER 
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Groups 


Mean GPA SD df t p 
Residential--Fall 
.. 0000 0. 7317 
EE1, EE2, ЕЕЗ, EE4, EES 2. 00 " үе" hues 
RC 2. 2188 0. 6664 
Residential--Winter 
ЕЗ, EE4, EES 2. 2098 0. 6655 
SPAN RR RER 97 1.3690 n. S. 
RC 2. 0038 0.8219 
English--Fall 
EE1 1.5237 0. 4158 
22 -2.2556 n. 5. 
ECI 2. 1244 0. 6885 
EE3 2.0517 0. 7839 
25 0. 2068 n; 8. 
EC3 1. 9887 0. 7889 
ЕЕ5 2. 5600 0. 6686 
22 1.5982 n. S. 
EC5 2. 0194 0. 7826 
English--Winter 
EEI 1, 7762 0. 7509 
22 -1.3495 n:s, 
EC1 2. 2031 0. 7208 
EE3 2. 1567 0. 6437 
25 0. 9439 n. 5, 
EC3 1, 8380 1.0156 
EES 2.5914 0. 6575 
22 1. 0109 n. 5. 
ЕС5 2.1918 0. 9505 


willassume they did n 
Inthree of the five groups, however, they were twice 


as large as the experimental group, and, therefore, 
would have no reason to feel like outsiders, 


No difference between the ex; 
trol groups on GPA su) 
by Stakenas (9). 


‘perimental and con- 
pported the results found 


No explanation for the higher percentage of men 
than women in the more satisfied group can be sug- 
gested from the results here. The fact that those 
more satisfied with their college life spend more 


time Studying is an interesting fact that might be 
worthy of future research, 


From the findings reported in this study as well 


as those reported by Stakenas (9), future солаіаа 
Чоп of the use of a cluster grouping technique prs 
Seem to be of little value as a means of кылаш 
better and faster Student adjustment to college. dee 
however, further study of cluster grouping were ox- 
sired, these Suggestions appear relevant: 1) D 
imity within the residential areas might prove cns 
effective than grouping around English. 2) Mem ry 
Ship ina clustered &roupshouldbe puton a voluit 
asis in order to maintain the groups. 3) A qud 
evaluation of courses Shouldbe made before they de 
included in the Clustered group. If at all possil the 
an attempt should be made by the instructors on as 
clustered courses to coordinate assigned materia the 
a means of enriching the overall coursework for sid- 
clustered groups, Althoughthis would involve e 
erably more time and effort on the part d is 
professors involved, this enrichment might S 
more students to become members, It is possi act 
however, that the enriched courses would am un 
only the brighter Students, not a representative £r 


LARSEN-MONTGOMERY 35 


TABLE 4 


PERSONAL INFORMATION DATA FOR ORDERED 
INDIVIDUAL SCORES DIVIDED INTO THIRDS 


Variable df x? 
1. Sex 2  6.2072* 
2. Size of high school 4 5. 2706 
, 3. Location of high school 4 0.2241 

4.  Fraternity-sorority 

affiliation 2 0. 2065 
5. О. Т. as first-choice 

college 2 2. 2969 
6. Knowledge of group 

membership 2 0. 6798 
7. Number of courses in 

schedule 2 2.5342 
8. Residence (home vs, other) - 0. 00008 
9. Number of visits home 4 2.2744 
10. Amount of time spent 

ч studying 10 20.9559** 

11. Amount of time spent 

in recreation 6 1. 9348 
12. Amount of time spent 

socially 6 3.1186 
13. Amount of time spent 

working 0 0. 00008 
14, Amount of time spent 

otherwise 1 0.0168 
15. Group in which friends 

are listed 4 3.9176 


?Insufficient data. 
*Significant at .05 level. 
**Significant at .025 level. 
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TABLE 5 


PERCENTAGE OF MEN AND WOMEN IN EACH 
THIRD OF ORDERED INDIVIDUAL SCORES 


Ordered Scores Men Women 
Top third 29.2 40.4 
Middle third 30.2 38.6 
Bottom third 40.6 21.1 
Total 100.0 100.0 
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TABLE 6 


AL SCORES 
PERCENTAGE OF TIME SPENT STUDYING IN EACH THIRD OF ORDERED INDIVIDU: 


Percent of Time Spent Studying 


60 70-90 
Ordered Scores 10-20 30 40 50 
24,2 
Top third 69.6 30.8 30.4 37.5 17.6 
30.3 
Middle third 17.4 34.6 34.8 33.3 58.8 
45.5 
Bottom Third 13.0 34.6 34.8 29.2 23.5 
Total 100.0 100.0 100.0 100.0 100.0 100. 0 
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A STUDY OF SUPERVISORS’ RATINGS OF MOST 
EFFECTIVE AND LEAST EFFECTIVE 
INDUSTRIAL ARTS TEACHERS ON THREE 
COMPETENCY DIMENSIONS' 


MORTON MARGULES and VIRGINIA M. KEITH 
University of Ottawa 


ABSTRACT 


The problem was to determine the relationship between industrial arts supervisors’ judgments of teacher 
effectiveness and their judgments of teacher competency on the dimensions of interpersonal relations, teaching 
techniques, and technical proficiency. A device was developed to obtainthe supervisors’ perceptions of their 
mostandleasteffectiveteachers. Thedataobtained were used to determine the relationship of the judgments of 
effectiveness to scores on the dimensions. Analysis of variance and Scheffe' S-method were used to analyze the 
data. The conclusions were: teachers are judged more competent in technical proficiency; the least effective 
teacher has greater deficiencies in interpersonal relations and teaching techniques; and the teaching techniques 
dimension differentiates, to the greatest degree, the most from the least effective teacher. 


RESEARCH ON teacher effectiveness has frequent- 
ly made use of supervisors as raters. Yet, after 
ratings have been obtained for use in the study, the 
supervisors making the ratings have been forgotten, 
While teacher effectiveness has been the subject of 
multitudinous studies by Barr (1), Ryans (9), and 
others, a major facet lacking consideration has been 
the supervisor's approach to teacher evaluation. 


This investigator attempted to relate teacher ef- 
fectiveness, as viewed by the supervisor, to dimen- 
sions of the teaching process which were derived from 
an analysis of the instructional process by observa- 
tion and discussions with leaders in the field. Recog- 
nizing that the nature of the subject matter occasions 
differences in the teaching process, it was deemed 
advisable to consider teaching ina singlefield. There- 
fore, the particular field of industrial arts was selected, 


Industrial arts is considered a unique area of ed- 
ucation because behavioral changes in students are 
Sought by the use of course content and facilities which 
Closely relate to our highly industrialized society. 
In view of the emphasis being placed upon this area 
of education, it was appropriate to seek information 
Concerning the approach of the industrial arts super- 
Yasor to evaluating teachers; The studies of Ehrenborg 
use Schank (11), andothers, showed that supervisors 
Yet Modest of factors to assess teacher effectiveness. 
n would Seem reasonable that the industrial arts 
а i Sability to relate to others, his ability touse 
pro lety of teaching techniques, and his grasp of ap- 
MEL technical Concepts and skills are dimensions 

Shoulddetermine hiseffectiveness, Therefore, 
€ Purpose of this Study was to determine the degree 


to which industrial arts supervisors consider such 
dimensions as interpersonal relations, teaching tech- 
niques and technical proficiency, in their evaluation 

of industrial arts teachers and how these dimensions 

relate to judgments of teacher effectiveness, 


The scope of this study is limited by the following: 
(1) the approach to the complex problem of industrial 
arts teacher effectiveness is made in terms of the 
three dimensions stated; (2) it was not the purpose 
of this study to determine which of these dimensions 
is most needed for effective teaching. Thus, this 
Was not a study of the behavior of teachers but a stu- 
dy of the nature of supervisors’ perceptions of and 
judgments about the behavior of teachers, 


Significant contributions of this study may be: 


1, toshowa relationship between judgments of ef- 
fectiveness апа underlying generic dimensions, 


2. to encourage others to make comparisons 
across subject matter areas of the various 
dimensions to teacher effectiveness, 


3. to provide a basis for training supervisors to 
make objective judgments of teacher effective- 
ness, 


4. to provide a basis for the development of in- 
struments upon which Supervisors can render 
objective judgments of teacher effectiveness, 


5. tostimulate educators of industrial arts teach- 
ers to consider supervisor's expectations of 
teacher behavior. 
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RELATED RESEARCH 


A searchof the literature revealed no study which 
sought the global teaching dimensions which may be 
used by a supervisor when he makes ajudgment about 
teacher effectiveness. In the training of industrial 
arts teachers, desirable teaching techniques and tech- 
nical proficiency are an accepted part of the curricu- 
lum. Interpersonal relations, as such, are not 
formally included inthe teacher training program, 
Thus, it would seem that more weight would be given 
to technical proficiency and teaching techniques than 
to interpersonal relations when the supervisor is con- 
sidering teacher effectiveness. 


With the exception of technical proficiency, the 
other two dimensions stated above have been consid- 
ered by most studies, But technical proficiency is 
а most important ingredient when we consider indus- 
trial arts teacher effectiveness. It is even more im- 
portant when we consider its significance in the super- 
visor's criteriafor evaluating the teacher, No study 
has been undertaken which specifically uses the dimen- 
sions this investigator hasdevised to determine what 
considerations might apply when the supervisor of 
industrial arts evaluates teachers in his charge, 


HYPOTHESES 


The null hypotheses investigated in this study were: 
1. When the three dimensions of interpersonal 
relations, teaching techniques, and technical 
proficiency are considered cummul 
teachers rated most effective will not 
significantly higher overall ratings of 
tency than teachers rated least effecti 


atively, 
receive 

compe- 
ve. 


2. Allthree dimensions will 
cant in discriminating b 
and least effective ind 


prove equally signifi- 
etween most effective 
ustrial arts teachers, 


METHODOLOGY 


Sample 


The sample consisted of randomly selected indus- 
trialarts supervisors who had а minimum of 5years 
experience in this capacity and who, by virtue of their 


assigned location, were in daily contact with their 
supervised teachers, 


Preparing the Dimension Scales 


The three dimensions used were interpersonal re- 
lations, teaching techniques, and technical proficiency. 
The operational definitions follow: 

1. Interpersonal relations refers 
that ‘‘goes on” between 
(or others ) by way of 
understanding, and m 


to everything 
one person and another 
Perception, evaluation, 
ode of reaction 


2. Teaching techniques refers to the pur d 
involved in the action necessary to establis! 
a common bond between teacher and learner. 


3. Technical proficiency refers to the knowledge, 
Skill, and degree of expertness with which the 
principles of industrial arts, mechanicalarts, 
and sciences are applied. 


The following steps were taken to develop and vali- 
date the observation instrument: 


° Statements were gathered from the literature and 
the responses of the supervisorsto the questionnaire. 
Abstract, ambiguous, irrelevant, and blanket items 
were removed and the statements made objective, 
workable and behavior-oriented. 


о The remaining items were then submitted to a jury- 
This groupof judges was requested to eliminate those 
items which they felt had no value in determining in- 
dustrial arts teacher effectiveness in each of the 
three dimensions. They were then asked to rank 
order the remaining items on the basis of their value 


for ascertaining the most effective industrial arts 
teacher. 


* The ranked frequencies were then established, The 
mean for each itemwas calculated and all items 
were placed in a final ranking. A total of 109 items 
were derived--thirty-seven in both the interpersonal 
relations and technical proficiency dimensions and 
thirty-five in the teaching techniques dimensions. 


ө Тһе items were phrased so that half of the favor- 
able responses were in positive terms, and the other 
half of the favorable responses were in negative ter ms. 
It was expected that this procedure would discourage 
the development of a response set on the part of the 
supervisors. The items were placed in a Likert- 
type scale of one to five. The value of five was used 
to indicate the most effective category. The items 
were then randomly ordered in the pilot aii 
A random selection of ten supervisors were then as vs 
to rate their most effective and least effective teac 


re 
er. Effectiveness scores on twenty teachers wer 
thus obtained, 


: e 
eAn item analysis was conducted according to e 
Flanagan method (4) to determine the discriminator 


power of each item and to establish the internal vali 
ity of the device, 


of 

eAfter the item analysis was completed, a wl Я 

Sixteen usable items for each dimension were sele The 
lanagan correlations ranged from . 70 to . 87. 


А е 
final Schedule, was then developed by using thes 
items, 


* The final instr 
inalten su 
teachers, 


ument was given to each of the ony 
Pervisors witha request to rerate the xf the 
Four weeks elapsed between the use © ns 
Pilot instrument and the final instrument, The AU lke 
of the second testing were then correlated with t 


i e 
corresponding items on the original scale using th 
Flanagan method, 


e 
e Based upon the dual criteria of consistency on th 


nat each 
test-retest and the Flanagan г, two items in ea¢ 
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dimension scale were dropped due to low r’s. Corre- 
lationsonthe forty-two remaining items ranged from 
- 56 to. 87, 


eThe instrument was then sent to fifty-seven ran- 
domly selected supervisors who met the selection 
criteria. Fifty-two were usable for the study and 
provided a 91 percent return. The instrument pro- 
vided the raw scores for 104 teachers on the three 
competency dimensions. A total raw score for each 
teacher was obtained by adding the scores given by 
the supervisor on each of the competency dimensions. 


In order to assess rater validity, previous admin- 
istrators ratings of the participating teachers were 
correlated with the scores of overall effectiveness 
obtained from the supervisors. 


Analysis Plan 


A 2x 3analysis of variance with repeated measures 
on the factor was used to analyze the data. 


Three effects were tested: 
1. the main effect of effectiveness rating (A) 


2. the main effect of the specific competency 
area (B) 


3. the interaction between the effectiveness rating 
and the specific competency area (AB) 


For each significant effect the Scheffe’ S- Method of 
comparing means was used. 


The administrators’ ratings were correlated with 
the scores on the three dimensions individually for 
the effective and ineffective teachers separately, 
This was done to compare the discriminability of the 
administrators’ ratings with that of the supervisors, 


RESULTS AND DISCUSSION 
Table 1 provides the range and means of selected 


characteristics of the supervisors who participated 
in the study. If the supervisors in this study differ 


TABLE 1 


at allfrom the ordinary Supervisor, it is in the direc- 
tion of being more experienced, and perhaps, there- 
fore, more sensitive. The Table also suggests that 
the supervisors, asa group, have been somewhat 
prepared for their role as upgraders of instruction 
which, per force, includes evaluation. 


The results of the analysis of variance of effective- 
ness and competency dimensions appear in Table 2, 
TABLE 2 


ANALYSIS OF VARIANCE OF EFFECTIVENESS (A) 
AND COMPETENCY (B). 


Source of 
Variation 55 df MS Е 


Betweensubjects 51, 736 103 
А (effectiveness) 26, 017.66 1 26,017.66 103,18 
Subjects within 


groups 25,718.34 102 252.14 
Within subjects 1,915 208 

B (competency) 1103.82 2 351.91 74.01 
AB 213.48 2 106.74 21.80 
Bx subjects 

withingroups 997.70 204 4.89 


B.J. Winer, Statistical Principles in Experimental 
Design, New York, McGraw-Hill, 1962, p. 307. 


Main Effect of Effectiveness 
2n "ect of Effectiveness 


The main effect of the effectiveness rating was 
significant (Е = 103,18, p<, 001). An examination 
of Table 3 shows that the most effective teacher group 
had overall higher ratings across the three dimen- 
sions than the least effective teacher group. This 
was expected in view of the fact that the supervisors 
had a priori made the effectiveness selection and, 
therefore, were likely on any evaluative item to indi- 
cate their effective teachers were superior to their 
ineffective teachers, 


RANGE AND MEANS OF SELECTED CHARACTERISTICS OF SUPERVISORS (М = 52). 


Years of Professional Supervision 
Experience 
———————— Time No. of 
Age Education? Teaching Supervision Availableb Teachers 
Range 30-66 2-8 3-36 4-30 0-8 2-19 
Mean ат, 42 5. 06 14, 35 1.92 3.21 6.35 


a E i 
ducation - 4 = Bachelor degree 
5 


i 6 = Sixth level equivalent 
b T Я year level equivalent 
ime available for supervision during the day. 


Master's degree or its equivalent 


40 THE JOURNAL OF EXPERIMENTAL EDUCATION 
TABLE 3 


SUMMARY TABLE FOR TEACHER EFFECTIVENESS (A) AND TEACHER COMPETENCIES (B) MEAN SCORES 
(п=52). 


Competencies 
Effectiveness Interpersonal Teaching Technical а 
Rating Relations Techniques Proficiency um 
bi b2 b3 
Most Effective 62, 6а 62.3 64.0 63.0 
ay 
Least Effective 44,6 41.9 47. 6 44.7 
ag 
Sum 53.6 52.1 55.8 


Note: All cell means differ Significantly from one another (except between cells a, by and a, bg and between 


а] bj and ay b3) by the Scheffe’ S- method, # Except for between cells a1 bg and aj bg which differ at the. 05 


level, all other Significant differences attain the . 01 level of confidence, 


a George А. Ferguson, Statistical Analysis in Psychology and Education, New York, McGraw-Hill, 1966, p. 296. 


The great difference in means, however, suggests 


methodology of teaching industrial arts subjects, ар” 
that there is a wide differentiation being made be- parently are not adequately designed to provide the 
tween these two effectiveness groups of teachers by teaching techniques which Supervisors deem neces- 
the supervisor when he evaluates their behaviors ja sary for their industrial arts teachers, Or perhaps 
terms of an overall effectiveness Score. Therefore, the instructional methodology provided at the college 
industrial arts supervisors do discriminate signifi- is more in tune with contemporary thinking, and the 
cantly between the most effective teacher and the Supervisors аге not familiar with the latest methods 
least effective teacher when the three competency ш 


of lesson pre i 
dimensions of interpersonal relations, t presentation, 


eaching 
techniques, andtechnical proficiency areconsidered It is also sug, ver- 
z 5 gested, from a perusal of the o & 
asacomposite. On this basis, therefore, null hy- all means onthe Competency Бекем es that indus 
pothesis one was rejected at the . 01 level of confi- trialarts teachersas t 
dence. 


agroupare seenby their super 
atively more effective in dealing ii 
in presenting instructional mate 
Suggest that industrial arts teacher 


visors as rel 
Others than 


Main Effect of Competency Dimensions Subtly, it may 


m Ho ErOUD аге less articulate in presenting instruc” 
The main effect of the specific competency scores tional material where iii п — i verbal 

(B) was significant (F-74.01, p<, 001). It was Communication are factors 

Shown that supervisors of industrialarts considered d 


both the most effective and the least effective teacher It у 
groups as being significantly more competent in tech- dimensions interpersonal relations would be signifi" 
nical proficiency than in either interpersonal relations cantly lower than the Е 
or teaching techniques. This was somewhat expected group. Thi i lac 
when the неон ОЁ industrial arts teachin edu- PricanmPtion was о везови 
cation institutes is considered, Approximately one- 
third of the courses given are devoted to providing 
experiences designed to lead to technical competen- 
cies in the subject matter areas which comprise in- 
dustrial arts education, 


Was anticipated that among the three competency 


more oriented toward objects n 
ess oriented toward people. ch- 
e case for the least effective tea 


It was unexpectedthat among the three competency 
dimensions teaching techniques would be significantly 
lower than technical proficiency inthe most effective 


effective teacher 
teacher education 


petency dimension means differed 
he . 001 level of confidence. 


A 


MARGULES-KEITH 41 


Interaction Between Effectiveness and Competency 
Dimension 


The interaction between the effectiveness rating 
and the specific competency dimension (AB) was sig- 
nificant (F-21.8, p< .001 ). The data presented 
show that the most effective teacher group was far 
superior to the least effective teacher group on the 
three dimensions. The mean differences between the 
two effectiveness groups were extremely large for 
each dimension, However, relatively small differ- 
ences were shown between competency scores within 
each group. Infact, between cell differences within 
each effectiveness group were numerically only 13 
percent as large as mean differences between effec- 
tiveness groups. It appears that the sensitivity of 
the supervisor is small within effectiveness groups, 
but extremely large between the groups. The data 
suggests that it might be appropriate to repeat the 
study using afullrange of teachers instead of two ex- 
treme effectiveness groups. This method might show 
less gross differences between effectiveness groups 
and greater differences within effectiveness groups 
on the three competency dimensions, 


In judging the most effective teacher, only the 
competency dimension of technical proficiency was 
differentiated. A difference between interpersonal 
relations and teaching techniques was not obtained. 
This finding suggests that competency in interperson- 
al relations may be closely related to competency 
in teaching techniques when the most effective teach- 
eris considered. Competency in teaching techniques 
may be related to the disirable interpersonal relations 
exhibited by the teacher. This may then lead to the 
point of view that highly desirable interpersonal re- 
lations are necessary to present instructional mate- 
rial effectively. In addition to this point of view, it 
can be inferred that the supervisor cannot tell the 
difference between interpersonal relations and teach- 
ing techniques when the most effective teacher is ob- 
served in action. It may be conjectured that in eval- 
uating the most effective teacher, the supervisor 
merges all dimensions into the overall effectiveness 
rating, whereas in evaluating the least effective 
teacher, he uses more care in trying to determine 
the reasons for ineffectiveness, 


The lowest mean for the least effective teacher in 
teaching techniques was quite unexpected. The addi- 
tional courses in teaching methods required of indus- 
trialarts teachers purport to developan understanding 
of the principles of teaching so that a variety of tech- 
niques, methods, and approaches may be adapted to 
effective teaching. One might conclude that what is 
gained from methods courses is not commensurate 
with the time spent on them. However, this conclu- 
Sion shouldbe tempered, because we are discussing 

е less effective teachers. The same results were 
not obtained with the more effective teachers, 


Differences among the three dimensions were evi- 
nthe supervisors judged their least effec- 
Chers. Least effective teachers tend to be 
in inter, less competent in teaching techniques than 
One ехррег50па] relations and technical profic iency. 
techni anation may be that industrial arts teaching 
than E are more easily observed and classified 
teache. w personal relations. For the ineffective 
9 is ineffective in all areas, it may be 


easier for the supervisor to perceive the more visi- 
ble dimension of teaching techniques. It may also 
be that a modicum of technical proficiency cannot 
overcome deficiencies in teaching techniques, 


It is apparent from the data that technical profi- 
ciency is not an adequate basis for considering a 
teacher effective in industrial arts. It is probably 
true that when evaluating teachers, supervisors per- 
ceive the more tangible outcomes of technical profi- 
ciency quite easily. Their perception may be enhanced 
by the fact that industrial arts teachers are, rightly 
or wrongly, considered to be technicians with highly 
developed technical abilities and skills. However, 
since the content and concepts of industrial arts are 
becoming more complex and involved due to the con- 
temporary technology, supervisors may be viewing 
the teaching of industrial arts in new terms. They 
may now be looking for competencies other than those 
strictly manipulative acts and immediately discerni- 
ble related knowledges. They feel rather that these 
other competencies should be part of the professional 
teacher’s repertoire. The teacher needs to under- 
stand the basic principles of teaching and learning 
and he must be able to organize learning experiences 
by using a variety of methods appropriate to the lesson 
at hand, 


It can be concluded that the most effective indus- 
trialarts teacher has exceptional competencies in the 
areas of interpersonal relations, teaching techniques, 
and technical proficiency. But of more concern, it 
can be concluded that the least effective industrial 
arts teacher has deficiencies in all three dimensions, 
but is particularly deficient in teaching techniques, 
It was shown that supervisors’ scores for the least 
effective teachers featured greater discriminability 
among the three competency dimensions than is true 
for their scores for the most effective teachers, 


Relationship of Supervisors’ and Administrators’ 
Judgments 


A marked significant correlation, Table 4, was 
obtained between the supervisors’ total scores and 


of the correlations for each dimension for each of the 
effectiveness groups separately did not yield totally 

valid correlations. This was attributable to the use 

of only one-half of the total effectiveness data in com- 
puting such correlations, Therefore, the correlation 
within each cell was Spuriously low, However, one 
can interpret the relative sizes of such correlations, 


From these correlations, it may be concluded 
that supervisors and administrators are most inclined 
to agree on the dimension of teaching techniques as 
concernsthe least effective teachers, Thence, more 
evidence for the visibility of this dimension among 
poor teachers, Of note also isthe fact that agreement 
within cells is highest for the most effective teacher 
ontechnical proficiency , While for the least effective 
teacher agreement is highest on teaching techniques, 
Thus, a teacher is probably judged most effective 
because of his high technical proficiency, 


Itisalso suggested from this analysis that indus- 
trialarts supervisors and administrators, although, 
significantly in agreement on their overall evaluation 
of industrial arts teachers, may be perceiving a dif- 
ferent set of behaviors exhibited by the teachers when 
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TABLE 4 


PEARSON PRODUCT-MOMENT CORRELATIONS BETWEEN SUPERVISORS’ COMPETENCY SCORES AND 


ADMINISTRATORS' RATINGS CLASSIFIED BY SUPERVISOR'S EFFECTIVENESS RATINGS (n - 52) 


Competencies — 
= ical ы 
i Interpersonal Teaching Technica Е 
е 
ae Е Relations Techniques Proficiency Score 
Bi bo Bg 
.421 
Most Effective . 293 . 383 . 505 
ay 
. 493 
Least Effective .333 .499 .458 
a 
733 
Total Score? .635 ‚134 .663 . 


Lawrence T. Dayhaw, The Dayhaw Correlation. Chart, Ottawa, Canada, 


a № = 104, 


Note: All correlations are significant at the ‚ 01 level of confidence excepta, b 


these three dimensions are considered, Orthey may 
be reacting to them differently. It further suggests 
that the technical proficiency dimension has relative- 
ly high visibility for both the supervisor and the ad- 
ministrator, particularly when compared to inter- 
personalrelations. This may be due to the concrete 
nature of technical skills and knowledges. When 
considering interpersonal behaviors and teaching 
techniques, most administrators bring to bear an 
academic background and, consequently, an orienta- 
tion somewhat different from the industrial arts 
trained supervisor, In Spite of this difference in 
background, however, one is Struck by the agreement 
rather than disagreement between judges. Allcor- 


relations were significant at the . 05 level of confi- 
dence, 


CONCLUSIONS 


This study has attempted to investigate the role 
of three competency dimensions in supervisors’ as- 


gnized as сотре! 


teaching process. The results seem to lead to the 
following conclusions: 


l. A study of supervisor 


A y practices in evaluating 
teachers is feasible, 


2. Positive relationships exist between certain 
generic teaching dimensions and supervisors’ 
judgments of teacher effectiveness, 


3. The three competency dimensions, interper- 
sonal relations, teaching techniques, and technical 
proficiency, do differentiate between the most ef- 


fective and the least effective industrial arts 
teachers, 


University of Ottawa Press, 1956. 


1: 


4. The least effective teachers have deficiencies 
on all three dimensions, but greater deficiencies 


in both interpersonal relations and teaching 
techniques. 


5. The three dimensions do not equally differen- 


tiate between most effective and least effective 
industrial arts teachers. 


8. Effective industrial arts teaching requires 
more than technical proficiency. 


-eat- 
1. The teaching techniques dimension to the Eom 
est degree differentiates the most effective {г 
the least effective teachers, 


8, Supervisors and administrators are py 
influenced most by high technical proficiency W d 
rating an effective teacher. On the other E 
they are primarily influenced by interpersona ing 
lations and teaching techniques when determin 

the less effective teacher, 


9. This approach to understanding the basis E 
which the Supervisor evaluates his teachers i 
the considerations upon which his judgments Ai- 
founded does not necessarily exclude the poss! or- 
tythatother dimensions are relevant and i Pdf 
tant. However, both thedata and logic of the Rees 
Support the assumption that the three dimensio! 


в 1 e 
Studied are influential factors in ratings of ette 
tiveness, 


РЕА 
10. Expectancies of supervisors and айй 
tors must be considered in teacher evalua 


3 a7 
While they tend to agree, they do not show CO? 
plete overlap. 


The findings of this Study may also: ro- 
l. Reflect a need for teacher improvement P 


a 
grams in teaching techniques and interperson 
relations, 


ў 
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2. Provide a basis for training supervisors to 
make objective judgments of teacher effectiveness. 


3. Provide a basis for the further development 
of instruments upon which supervisors can render 
objective judgments of teacher effectiveness. 


4. Stimulate industrialartsteacher educators to 
consider supervisors' expectations of teacher be- 
havior, 


5. Make feasible a comparison across subject- 
matter areas of the relative contributions of the 
various dimensions to teacher effectiveness, 


RECOMMENDATIONS 


On the basis of the findings of this study it seems 


appropriate to make several recommendations: 


1. Since very little research has focused on the 
problem of supervisory evaluation in industrial 
arts, it is suggested that within the basic frame- 
work of evaluation, further exploration be under- 
taken using the dimensional approach. The 
approach might also be explored in other subject- 
matter areas, 


2. The device used in this investigation should be 
reviewed and refined, 


3. Any new study undertaken should include a full 
range of effectiveness measures, so as to provide 
more refined data, 


4, The recommedation made within and without 
education that the role of teaching techniques and 
methods be reduced received little support in this 
study. A serious mistake, at least with regard 
to industrial arts teaching, would be made in re- 
laxing instruction in the teaching techniques area. 
Therefore, it is recommendedthat more attention 
be given to those courses which provide a vital 
understanding of the processes of learning and 
the methods of presenting industrial arts instruc- 
tional material, 


5. Exploration should be undertaken to seek ways 
of formally structuring into the teacher educa- 
tioncurriculum programs which are intended to en- 
hance positive interpersonalrelations. 


6. Finally, it is recommended that graduate level 
Courses in evaluation be developed for present 
andfuture supervisors of industrialarts, These 
Courses should have as their prime purpose the 
Presentation of the latest information concerning 
Subjective and objective teacher evaluation. This 
18 Particularly vital at this time in view of consid- 
eration being given to a shift from the guaranteed 
yearly increment to merit salary increases; ob- 
Jective evaluation must be ona firm and fair basis, 


easily understood by both the teacher andthe 
evaluator, 


10. 


FOOTNOTE 


. This manuscript is based on the writer’s doctoral 


dissertation completed under the direction of 
Professor Virginia Keith, University of Ottawa, 
Canada, 1968. 


REFERENCES 


Barr, Alvin S., *“Тһе Measurement and Pre- 
diction of Teaching Efficiency: A Summary of 


Investigations,’’ Journal of Experimental Edu- 
cation, 16:203-283, June 1948, 


ces 


Brookover, Wilbur B., ‘‘The Relation of Social 


Factors to Teaching Ability," Journal of Experi- 
mental Education, 13:191-205, June 1945. 


Ehrenborg, David, The Validation of a Device 
for Identification of Industrial Arts Teaching 
Success, unpublished doctoral dissertation, Los 
Angeles, University of California, 1963,xii- 307 p. 


Flanagan, John C., **General Considerations in 
the Selection of Test Items and Short Method of 
Estimating the Product- Moment Coefficient at 
the Tails of the Distribution," The Journal of 


Educational Psychology, 30:674-680, December 
1939. 


Giachino, J. W., An Analysis of the Success 
Qualities That Should be Emphasized inthe Train- 


ingof Candidates to Become Competent Teachers 
of Industrial Arts, unpublisheddoctoral disser- 
tation, University Park, Pennsylvania State 

University, 1949, xi-205 p. 


Medley, Donald M., and Mitzel, Harold Be, 
“A Tentative Framework for the Study of Effec- 


tive Teacher-Behavior,’’ Journal of Experiment- 
al Education, 30:317-320, 1962. 


Monroe, H.B., Personality Variables of Indus- 
trial Arts Teachers, unpublished dissertation, 
Greeley, Colorado, Colorado State College, 
1960, xiii- 78 p. 


Morgan, J.B., Personality Variables of Indus- 
trialArts Teachers, unpublished doctoral disser- 
tation, Greeley, Colorado, Colorado State College 
1962, xi-135 p. | 


Ryans, David G., ‘‘Assessment of Teacher Be- 


havior and Instruction," Review of Educational 
Research, 33:415-441, October 1963. ^ — 


Ryans, David G., ‘‘Theory Development and the 
Study of Teacher Behavior," The Journal of Ed- 


ucational Psychology, 47:462-475, December 
1956. 


44 THE JOURNAL OF EXPERIMENTAL EDUCATION 


i А 1954. 
11. Schank, Kenneth Lloyd, Industrial Arts Super- Education, 23:99-131, December 
vision: A Synthesis of Selected Supervisory Prin- 
i published doctoraldisser- А ки sions 
E es a ot perum University of Maryland, 14. Solomon, Daniel, “Teacher Behaviors Dimen: , 
" А 


і valuations 
istics, and Student Evalua 
d я Course Characteristi 5, 


of Teachers," American Educational Research 
Journal, 3:35-47, January 1966. 
12, Selvidge, Robert W., “Success Factors in 
Teaching, ”” The Industrial Education Magazine, 
33: 305-306, June 1932, 


he 

15. Torrence, Andrew Pumphrey, “А pend дЕ. 
Relationships of Certain Competencies to Su nal 

in Teaching Vocational Agriculture," Journal 


13. Singer, Jr., Arthur, *'Social Competence and 


er 
of Experimental Education, 25:1-31, Septemb 
Success in Teaching," Journalof Experimental 1956, 


—= 


————M—— 
New Title From DERS 
— MR ENS. 


Basic to the improvement of е 
about the processes and conditi 


system is subje 
1L, five authors provide scholarly i 
Structures of such disciplines as mat 


English, and reading. They state or imply that the 
tracture of the disce. ial 0 lated by on > 
structure of the discipline itself, as formulate y one 
RESEARCH AND DEVELOPMENT or more scholas has provided much of the basis for 
recent curriculum devlopment, 
TOWARD THE IMPROVEMENT Instructional materials and media can be consid 
OF EDUCATION as the interface betwei 


en the learner and the subject 
malter being leamed. In Part Ill, the discussion 
‚у А OCuses on instructional televi 
Gred by Herbert J. Klausmeier and George T. of the status, defects ang 
search, and on computer base 
$5.75 softcover methods, or the interactions which occur between a 
teacher and students, have been the subject of many 
In this book 20 eminent scholars, researchers, and — research studies in the last decades. In Part IV, two 
research. administrators present thei recent paradigms for research on teaching are pre- 
about the processes and cond sented. One approach discusses what is commonly 
the processes and Programs of instruction, and the called micro-teaching and the other is interaction 
icat ind devlopment Strategies to analysis. 
educational problems, Finally, in Part V, three research administrators 
As the editors point out in tj 


A discuss the current a plication of research and de- 

tion of research and development strategies for the velopment strategies for the improvement of educa- 

Improvement of educational Practices is a pioneering tional Practices. "The 

Cure begun in 1964 under the provisions of the R & D is one of rapid 
ch Program of the United States by the ad 

Office of Education i 


ion, on some aspects 
5 of instructional re- 
d instruction. Teaching 


176 pages 


of these changes with reference to federal involve- 
E mentary and Secondary Education Act ment is traced їп one chapter. In another chapter the 
of 1965. Working from a proposed model outlined m — 1 the Wisconsin Renal and 
Chapter 16, the editors h Cognitive Learning are de- 
scribed. And in the final 


chapter, an author presents 
an output oriented model of R & D. 


Dembar Educational Research Services, 


Inc. Box 1148 
Madison Wisconsin 53701 


X 


THE JOURNAL OF EXPERIMENTAL EDUCATION 
(Volume 37, Number 4, Summer 1969) 


INCREASED COMMUNICATION IN 
LANGUAGE LEARNING 


DIETER M. MESSNER 


St. Olaf College 
Northfield, Minnesota 


ABSTRACT 


There is little opportunity for the individual student to speak in class, The ratio of speaking time to the 
time of other language activities is improper. The speaking time of the individual can be prolonged many times 


over in a 2-way communication model, based on the principle of paired learning wit 
and the teacher acting as a supervisor with special functions, 


h planned partner rotation 
This communication model provides the individu- 


al learner with a maximum of speaking time under controlled conditions, 


FACILITY IN extemporaneous speaking of a 
foreign language seems to be an unattainable goal for 
classroom language learners. The individual student 


has little chance of conversing in the foreign language. 


Only two people--the teacher and one student-- speak 
atone time in a real dialogue situation, while the 
others have to be silent and listen, Because of the 
usualstudent: teacher ratio, the opportunity for each 
student to participate in a dialogue is limited, 


DIAGRAM 1 


Diagram 1 shows the traditional predominantly one- 
way communication structure inclass, atthe moment 
of a teacher-student interaction (A= B. 


u) 


We seem to learn language most economically in 
conversationalcommunication, Dialogue is consid- 
ered to be the essence of language, realizing its main 
functions. 


Since spoken language is a motor activity as well 
as sensory process, the length of the speaking act, 
the number and frequency of motor stimuli sent from 
the speech organs to the brain, must be decisive for 
its learning. The intricate mechanics of speaking 
cannot be well- learned by merely listening. A certain 
amount of nerve-cell training and conditioning time 
is necessary for the learning of any skill, 


Let us compare spoken language, a verbal com- 
munication skill, with a dual sport. We imagine a 
beginning tennis player, who, every second day, is 
given a chance to volley five times with the tennis 
instructor. When alone he is supposed to learn the 
game out of a book and by practicing against a cement 
wall. Letustransfer this situation to the classroom, 
Theinstructor standsonone sideof the net; the other 
Side is crowded with the multitude of students, Al- 
though they have the possibility to learn from an ex- 
pert, the single learner rarely has the chance to play 
the **champion" teacher, The learners are not given 
the chance to play each other, Fortunately, tennis 
is not learned in sucha way as, during practice hours, 
the ‘‘students”’ do practice with each other. 


For the good performance of a Skill, the respec- 
tive chains of nerve-cells in the motor cortex have 
to be ‘tuned in’ frequently for a certain length of time, 
Considering the importance of the length of the Speaking 
process in language learning, the ratio of each learn- 
er's speaking time to the time spent on other activi- 
ties in class, like listening to, reading, or writing 
the language, is improper. Several language teachers 
report that the individual student has the chance to 
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participate in classroom conversation for an average 
of 1 minute or, at best, 2 minutes per lesson. In 
the ninety lessons of a school year, this would be 
180 minutes or only 3 hours of living speech. 


The tape laboratory has endowed the foreign lan- 
guage learner withnew possibilities; its effectiveness 
is recognized. For listening and drill purposes it 
works effeciently, but it offers only *canned lan- 
guage,’ presents no visibles Speaking partner, tires 
the listener easily, and Supplies only audible stimuli. 


Also, the choral response technique prolongs the 
Speaking time of students and involves the whole class 
in the activity. But, like all drills, it suffers from 
a low degree of reality, is sometimes monotonous, 
and often does not give the student something mean- 
ingful to talk about. It would be desirable, to use 
exercises which involve the learner more personally, 
while at the same time permit the practicing of the 
linguistic structures which the teacher wants to drill, 


In order to increase the Speakin 
vidual learner and to involve hi; 
in theforeign languageas a life 
à different communication model and learning mate- 
rials, which can Satisfy these needs, T propose that 
directed exercises in dyads, or pairs 
learning partners, equipped with textbook dialogue 
and lists of questions and answers 
teacher for acertain class, could do the job. 


£ time for the indi- 
m more meaningfully 
experience, we need 


If the actual lesson offers a 
Bbegin to read the dialogue 
ible repetition they continue 
many parts of the dialogue as 
they can remember, Periodically, the teacher can 
hand out lists withquestions and mod 


ter learning. Pa 
off the list, but i 
through the text, 
tionnaire furnishe 
essential parts of a 
partner B answer 
printed material: 


uestion 
glanced 

he ques- 
swers or 
nswers(see appendix). If Possible, 
5 the question without looking at the 
or corrected 
This process 


DIAGRAM 2 
The learning pair or dyads in the classroom, Bj3- 
Bi47Big-formatriad. A, the teacher, is working 
with the pair В5-Вұ at this particular time. In con- 


isd racting i is 
trast to diagram 1, everyone is interacting in th 
communication structure, 


After a period of directed conversation based on 
textbook dialogues and question-answer lists, the 
teacher could Suggest continuing with extemporaneous 
Conversation, I have found that the din in the room 
hardly bothers the students, 


Multiple Preparatory and immediate controls of 
the questions and answers possible: 

1) preliminary oral preparation inclass, 2) prep" 
aration by the students at home, 3) immediato 
control by the learning partner, 4) the mode 

answers onthe list, and 5) the assisting teacher. 


This communication model could enable us to ee 
vert our classroom, whenever desired, into ‘‘lea à 
ing laboratories, " where every student is active a 
the same time and uses his time most profitably. ae 
ner learning, with the ideal ratio of оле CODE 
vides maximum participation for each (варгепово 4 
50 percent of the given time); it gives eye nile 
chance to speak and no one a chance to hide. W of 
the traditional communication structure in a pos in- 
twenty students would permit, for example, 1 could 
teractions within a given time, the dyadic mode К м 
Provide 500 interactions within the same pact ay 
experiences with this technique suggest that nied o 
be more effective to devote an entire class perio t 
it now and then instead of devoting short intervals а 
it every period, I dedicate every third class perio" 
entirely to this method, The ‘‘speaking program _ 
for this hour is Written on the blackboard: dd 
logues the students arc to practice, which pages of m 
question-answer sheets they are to go over and Dd 
often, and last but not least, a topic for extempor 
neous conversation is suggested. 


"n 


=y 


X 
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Since I cannot point to exact experimental results, 
the hypothesis hasthe status of an assumption. Class- 
roomobservations do, however, confirm its effective- 
ness. The learners soon gain self-confidence in 
speaking. The ''pathological" symptoms of people 
learning to speak a foreign language, hesitation and 
halting speech, disappear soon. Partners can dis- 
cuss common difficulties and can help each other. 
It is plausible that especially our late-starting lan- 
guage learners in high school and college should be 
given this opportunity for facilitating motor activity. 
Partner rotation provides the advantages of the larg- 
er group: a wider scope of individual responses and 
different learning experiences. The classroom is 
now more social as compared to the traditional cen- 
tralized and authoritarian communication structure, 
whichis mainly one-way. There is а more personal 
contact with the teacher, who meets and works with 
only two students atone time. The number of students 
has little importance in this communication model, 
since any number can be broken down into pairs and 
one possible triad. This technique is analogous to 
the communicative verballearning process in every- 
day life. Thequestion-answer process is not a com- 
pletely realdialogue,yeta great deal of any dialogue 
is devoted to the asking and answering of questions. 
It has been said that the ability to ask and answer 
questions efficiently lies atthe root of communication. 
The sets of questions and corresponding model an- 
swers canbe consideredasthe smallestunits of split- 
up dialogues. There is an added responsibility for 
the teacher who does have to make up his own ques- 
tions. On the other hand, good class rapport can be 
promised, based on my own experiences. 


Besides the compulsory written final exam, an 
oral final is scheduled for each student, in order to 
boost the motivation for the learning of the spoken 
language. Enoughquestions and answers accumulate 
duringone semester to make a sufficientbasis for a 
meaningful test-dialogue. 


The question-answer process, as the interaction 
between two people, is the basis of programmed in- 
struction. It is my hope that inthe near future we will 
have a *'teacher- proof'' textbook for dyadic language 
learning, which will incorporate meaningful conversa- 
tional question-answer interaction, important lin- 
guistic structures, as wellas the most frequently-used 
vocabulary of the target language. Such a book 
would be practical in schools and could be used for 
the self-instruction of learning partners or for single 
learners, if tapes were provided. 


APPENDIX 
A: What is a traveler asked at the border ? 
B: He is asked: ‘‘Doyouhave anything to declare?" 
“Мау I see your passport ?''/*' Have you a vaccina- 
_ tion certificate for the dog?" 
A: (repeats or corrects) 
e Have you already been abroad? 
А: Yes, already several times. /No, not ever. 
1 (repeats or corrects) 
a there any mail for you yesterday? 
re eS, I got two letters and one postcard. 


(repeats or corrects) 


= 
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: Do I stand opposite you? 
: No, you are sitting next to me. 


(repeats or corrects) 


: Where would you travel, if you had time and 


money? 


: If I hadtime and money, I would go to Florida or 


Switzerland. 
(repeats or corrects) 


: Do you always smoke a cigarette after dinner? 
: No, I don't smoke at all, / Just sometimes. 


(repeats or corrects) 


What do you do when you feel dirty? 


: When L.., Itake a bath. 


(repeats or corrects) 


Through what did Dr. Christian Barnard become 
famous? 

Dr. Barnard became famous through his heart- 
transplant. 

(repeats or corrects) 


What will you do in the coming summer? 
Iwillwork inordertoearn money. / I willattend 
a summer school, 

(repeats or corrects) 


In what month were you born? 


: I was born in August. 


(repeats or corrects) 


: What can one eat with a spoon? 
: With a spoon one can eat, for example, Soup, 


ice cream, and pudding. 
(repeats or corrects) 


: With which words does one begin a letter? 


One begins a letter with the words: Dear Sir, or 
Dear Madame, 
(repeats or corrects) 


In the summer, did you live with your parents or 
with your grandparents? 


? In the summer I lived with an aunt. / I lived in a 


camp. 
(repeats or corrects) 


: Where do you get your hair cut? 


Iget my hair cutatthe barber shop. /...atthe 
hairdresser, 
(repeats or corrects) 


: Do you like to dance? 
: No, Lam not a dancer. / I never learned it. / Yes 
, 


especially the English waltz and the rumba, 
(repeats or corrects) 


: In thewinter, what does one put into the cooling 


water? 


: One puts antifreeze into the cooling water, 


(repeats or corrects) 


: What disadvantages does a trailer have? 
: It is difficult to maneuver on mountain roads, it 
, 


is often illegal to park and it is Subject to man 
restrictions. y 
(repeats or corrects) 
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TEACHER PLANNING OF INSTRUCTIONAL 
MATERIALS FOR LOWER-CLASS CHILDREN 


GEORGE MOVESIAN 
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ABSTRACT 


А abetical 
ass by the respective superintendents, using Alba Edwards, AIphabe cared 
Index of Occupations and Industries as a guide. When teachers of lower- and middle-class children 
in selecting materi i 


ailure to 
ve classes, no significant differences were found. The fa 


IT HAS become an educational axiom that the | 


select materials that would not be significantly dif 
good teacher is one who is aware of individual differ- ferent from materials selected by: (a) teac e dus 
ences among the students. Some of these differences middle-class schools and (b) independent judg 
resultfrom differing social class backgrounds, Each 


Second, when given a free choice of mateni 
teachers in schools of disadvantaged children M for 
select materials judged to be more üpproppfau t 
the background and experiences of middle-class 
Struction, based on an awareness of the character - dren then those of lower-class children. 


А пр- 
To test these hypotheses, the following assump 


F ex- 
rials, i.e, famil- tions were made: First, different social RIA 
iar, Tecognizable, and within the cultura] Context of ist within our American culture and schoolc sition. 
the student, A Progression from the known reflect, in various ways, the family class po upon 
to the unknown is essential, Most teachers know this, Second, the middle class hasa greater pae E of 
Almostall have been exposed to such ideas incourses the Curriculum of the sc hools than any combina the 
in education, However, do teachers use this know- the other divisions of the socialclasses. Third, 
ledge? Do they plan lessons which indicate an aware- 


11151 ir ex- 
Principle of teaching children by utilizing their 


ing- 
One Periential background to help make the Ra 
clue to the answers to these Questions may be dis- learning activity meaningful to the learner meum in- 
Covered inthe response to the question ‘When given tionally sound, Fourth, the principle of ЕЕ 
а free choice do teachers Select materials appro- dividual needs, wants, and interests in the p 

Priate for students in terms of their backgrounds ?* 


= ound, 
of teaching-learning activities is educationally 5 
It seems safe to assume that 


if teachers selected SAMPLE 
the appropriate materials there 


would be some ор- 


em to reflect апа adapt their teach- Sixty-one teachers were selected from a ae 
ing methodology to Specific backgrounds of the ir list of schools identified as either lower- or mi Ed- 
students, class by the respective superintendents, be ee 
j | wards, Alphabetical Index of Occupations and of 
This study attempts to investigate this question, tries (1) asa guide. The social class position a= 
Accordingly, the following hypotheses were tested: the student Population of each classroom was ae 
| fully investigated, Only those teachers whose s п = 
,. When given a free Choice of materials, teachers Populations contained at least 60 percent lower 
їп Schools of Predominantly lower-class children will middle- 


i ere 
class were used, Lower-class children w 
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A COMPARISON OF MIDDLE CLASS MATERIALS SELECTED BY TEACHERS OF LOWER- AND MIDDLE- 


CLASS CHILDREN 


Middle-class Materials 


Teachers of Lower-class Children obsvd. 
exped. 


Teachers of Middle-class Children obsvd. 
exped. 


B. ar = 1,6047 ө» Fail to Reject 


40-55% 56-66 % 67-100 % Totals 

12 11 28 
(6. 88) (11. 79) (9. 33) 

12 8 29 
(7.12) (12, 21) (9. 67) 

24 19 57 


*43 % was the lowest percentage of middle-class materials selected by both groups of teachers, 


those children whose parents, when gainfully e m- 
ployed, were classified as unskilled. Middle-class 
children were defined as those children whose par- 
ents, when gainfully employed, were classified as 
professional, proprietary, or managerial, except 
farm. The investigator obtained information from 
twenty-eight teachers of lower-class children and 
twenty-nine teachers of middle-class children, 1 


The teachers were located infive urban and three 
suburban communities, The eight separate school 
districts were within 25 miles of the center ofa 
large city. 


INSTRUMENT 


In attempting to have a common subject areafrom 
which teachers would plan a unit of work, a unit on 
transportation was selected because it is required 
by the New York State Department of Education, To 
allow for maximum freedom for teachers to select 
instructional materials for a unit of work, a resource 
unit titled ‘‘Transportation’’ for elementary-school 
children was constructed, This resource unit con- 
sisted of a carefully planned series of suggestions 
centered on the topic and organized with ideas, ma- 
terials, and procedures to help the teacher in pre- 
planning thelearningunit. The recommended books 
were measured by a jury of three judges according 
to the Movesian Reading Rating Scale of Class Cul- 
ture which rates books for their appropriateness for 
children of Lower- and middle-class backgrounds. 2 
The selection of the jurors was based upon their re- 
spective background which was consistent with this 
study, i.e., that they understand the problem of 
identifying instructional materials for lower-class 
children and are interested in the improvement of 
instruction for these children. 


At a personal meeting with the teachers, the re- 
Source units were distributed. The teachers were 
asked to plan a unit of work for their respective 
classes, At notime was social class mentioned. 


Each teacher had the freedom to select as many 
items from the resource unit as he thought sufficient 


for a teaching unit. Therefore, the number of books 
and specific titles selected varied from teacher to 
teacher. 


RESULTS 


For purposes of comparing the selections obtained 
from the teachers of lower- and middle-class chil- 
dren, a percentage of the total materials (books on- 
ly) defined as middle-class was computed. The per- 
centage score for each teacher was placed inthe ap- 
propriate cell of a two-by-three table (see Table 1). 
The Chi-square test of association was used to test 
the hypothesis of ‘‘no difference” in the materials 
selected by teachers of lower- and middle-class chil- 
dren, respectively. The hypothesis, which stated 
that given a free choice of materials, teachers in 
Schools of predominantly lower-class children se- 
lected materials that are not significantly different 
Írom materials selected by teachers in schools of 
predominantly middle-class children, was not re- 
jected. The 95 percent confidence level was used 
for all tests of significance. 


Each judge independently rated the books accord- 
ing to Movesian Reading Rating Scale of Class Cul- 
ture, Only those materials agreed upon by the entire 
panel of jurors to be on the same side of the Scale 
were included in the unit, Thus, all three judges 
ed e ED dt Ber opinion, the item was either 
lower- or middle-c ori i its i i 
Vip ded riented prior to its inclusion 


Totestthe subhypothesis, the data was treated in 
order to compare the differences of means: і.е, а 
comparison of the scale means of the judges and the 
teachers of lower-class children. The scale mean: 
for each of the entire listing of ninety-six books vare 
computed from the average score assigned to each 
book by the judges. The mean of the scale scores of 
all ninety-six books included inthe unit was computed 
to be 4.962. This was then considered to be the mean of 
the scale scores for the judges. The t-test of Inde- 
pendent Samples was used (see Table 2), 
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TABLE 2 
A COMPARISON OF MEAN SCALE SCORES OF THE JUDGES AND TEACHERS OF LOWER-C LASS 
CHILDREN 
V N 
Mean of the Computed Degrees of F-Test Ni 5 
Scale Scores t-value Freedom Variance 
Teachers 4. 720 РЕР ds T" T 8 
Judges 4. 962 


аваг HA; р - peo J FaiioRejet,o «26457 


Fest НА; p Py = 0 ee Fail to Reject 


of variance; therefore the hypothesis of **no differ- 
ence” between the selection of materials by teachers 
of lower-class children and the judges was not re- 


materials, and the t-test of “по difference" between 
the teachers of lower-class children and the judges 
was not rejected, the findings Supported the second 
hypothesis, That is, given a free choice of materi- 
als, teachers in Schools of predominantly lower- 
class children select materials judged more appro- 
priate to the background and experiences of middle- 
class children than those of lower-class children. 


The teachers of lower-class children chose a 
greater percentage of materials judged appropriate 


The mean scale Score 
of allthe books selected for each group was compared 
(see Table 3. The hypothesis of “no difference” 
was not rejected for either the t- or F-tests, 


Although the teachers of lower-class СНЛ 8р Е 
lected a greater percentage of the materials ju Ва 
to be more appropriate for middle-class enn 
than teachers of middle-class Children, the Moves s 
Reading Rating Scaleof Class Culture score of od 
selections was below those of the teachers of mid а 
classchildren, The teachers of middle-class childre 
selected more books witha higher scale score than the 
other groupof teachers, 


DISC USSION 


Teachers not only seem to ignore or are ignorant y 
of lower-class materials, but also do not seem ve >” 
acknowledge or acceptlower-class culture. There 
fusal of teachers and Schools to accept much of the 
language andovert behavior of lower-classchildren, 
concomitant with the selection of materials mere 
appropriate for the middle-class child's background, 
Suggests that teachers do not accept nor acknowledge 
lower-class culture, 


Reluctance on the part of teachers to use appro- 
priate materials for lower-class children may 2180 
Stem from the following reasons: (1) It may be tha 
the teachers feel lower-class children should aspire 
to middle-class values, They may judge that dus 
“Rising” by using the materials which were dee me: 
appropriate for middle-class children. (2) It may 


TABLE 3 
A COMPARISON OF MEAN SCALE SCORES By TEACHERS OF LOWER- AND MIDDLE-CLASS CHILDREN g 
Teachers of Teachers of Computed F-test of Ny "à 
Lower-Class Middle Class t-value Variance 
Mean Scale 
Score 4. 7195 4. 1662 .345 1. 383 28 "B 
o=, 135 
55df N 28 N 29 Hy H 
A: Pp - py =0 A: ру - Pg = 0 


e% Fail to Reject fe Fail to Reject 
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oy". 
be that teachers feel social class membership plays FOOTNOTES 
an insignificant role in growth and development, es- 
pecially in the area of instructional materials, i. e., 1. Four returns were deemed unusable. Two teach- 
books for children. In essence, teachers do not ap- ers of lower-class children refused to com- 
pear to be meeting the lower-class child's need for plete the instrument after agreeing to partici- 
instruction geared to individualdifferences based on pate. One teacher of lower-class children be- 
social class. came seriously ill and took a leavefrom 
his teachin: sition. Because one teacher of 
CONCLUSIONS mde clues children selected all ninety -six 
П " books, the teacher really did not discriminate 
From the data tabulated, it was found that when in his selection of books. Thus, the information 
givenafree choice of materials, teachers in schools gathered came from fifty-seven teachers. 
of predominantly lower-class children selected ma- 
terials that are not significantly different from the 2. Becausethecriterion of occupations of parents 
materialsof (a) teachers in schools of predominantly was used to identify the social class position 
middle-class and (b) the jurors rated and pre- of thechildrenfor whom the unit was intended, 
2 selected. the same criterion,i.e., occupations, wasused 
ш asthe major basis to determine which instruc- 
Second, when given a free choice of materials, tional materials were appropriate for lower- 
teachers in schools of predominantly lower-class and middle-classchildren. Because most ma- 
children select materials judged more appropriate terials investigated had some overlapping of 
for the background and experiences of middle-class Social class orientations (in terms of occupa- 
children than those of lower-class children. tions), the jurors were askedto evaluate each 
i of the materials in terms of a 7-point scale. 
These conclusions suggest that further research The numbers one, two, and three were used 
be undertaken testing the following hypotheses: for'thiowe materials judged to be appropriate 
(1) When given a free choice of materials, ele- es D 08 opal tae 
mentary school teachers of lower-class children do Edi E М es а 
a P : : с judged appropriate for middle-class children. 
not differentiate between materials judged appropri- A à ; 3 
i H ny materials which were judged as equally 
ate to be used by lower-class children and materials suitable for both lower- and middle-class 
judged Г Бе нөнү Diba tae ise Чыры. orients children (number four) were not included in the 
erately ignore using such materials. resource unit. Only those materials agreed 
i e same side 
Í (2) Itseems that teachers of lower-class children E dde dicet enim Feria sia 
А did not appear to meet the lower-class child's needs 
f of individual differences on a social class basis in 
the pre-planning of instructional materials. 
REFERENCE 
(3) Materials which could be judged appropriate for 
lower-class children were few when compared to ma- 1. Alba Edwards, Alphabetical Index of Occupations 
terials which could be judged appropriate for middle- and Industries, U.S. Department of Commerce, 
class children, Washington, D.C., 1950. 
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READING INSTRUCTION AND COUNSELING 
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ABSTRACT 


" as 
Ninety beginning freshmen at the University of Missouri-Rolla were utilized as subjects. The sample w 


Sub-divided into three equal groups; the exp 
in an intensified instructional program in Study Skills 


iz 


ion ta texdition" 
oup was accorded no variation in traditi 


e 
All three groups were matched on their obtained School and C o11e 
Ability Test Scores, English and Mathematics Placement Tesi 

Ability Test mngiish 


attempted. All students in these groups 
skills, 


The experimental group met for developmental readin, 


for no academic credit. The study skills inst: 
its and study attitudes. The developmental readi 


PRESENTLY, the demand for competent scientists 
and engineers is increasing while college enrollment 
in these scientific and technological areas is decreas- 
ing. One of the resulting challenges placed upon 
institutions serving the technological clientele is to 
provide additional special student services that will 
help insure graduation of the largest possible num- 
ber of well-educated and socially adjusted scientists 
and engineers, 


In a report (1) entitled, **Factors Influencing En- 
gineering Enrollment, ” published by the American 
Society for Engineering Education and Supported by 
the National Science Foundation, the statement was 
made that'*,, , engineering enrollments over thecoun- 
try are down... despite the coincidental rise in the 


college population. ” Further, this report recom- 
mends that 


:-.8 major effort be made to insure that a 
larger fraction of the Students who enter en- 
gineering Successfully complete the de gree 
requirements, If engineering attrition were 
reduced to the level of 1950, the effects would 

е the same as though freshmen enrollment 
Were increased by almost 29 percent. In 
1950, one hundred Were required to gradu- 
ate about 63 engineers; in 1959, approximately 
129 freshmen were required to produce 63 
engineering graduates, 


5 
t scores, college majors and semester hour 


ably lower for the experimental group than for the control groups. Highly 


reading comprehension. 


After considering the attrition among first E 
ter engineering freshmen at the University of W В 
Consin (2) in 1963, a conclusion was that RE 
for remediation of dificiencies in previous tean ee 
such as remedial study skills clinics might be Rm 
taken by engineering colleges desiring to ERE 
attrition of prospective engineers,’ This зан read- 
further supported the need for a developmenta of 
ing and Study skills program at the University 
Missouri-Rolla (UMR). 


Numerous psychologically oriented GONE. 
Centers have been provided to assist students athe 
their educational, vocational, and personal E cias. /* 
and adjustment. Fewer colleges and pde skills 
have provided specialized instruction in SS “only 
and developmental reading for their student da com- 

a limited number of institutions have ot mental 
bined program of Study skills and dev ee ле for 
reading instruction with psychological een higher 
their students, Because few institutions Tod asthat 
education have as homogeneous a student esearchers 
at the University of Missouri-Rolla, the lined pro- 
desired to investigate the effects of a coml ding in- 
gram of study skills and developmental Vue a selec- 
Struction with psychological counseling зт». 

ted sample of engineering and science stu 


PROBLEM fold: 
The purpose of the investigation was three! 


bU 
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1. toascertain the effect of a specialized 
program of study skills, developmental 
reading, and psychological counseling on 
the obtained GPA of the experimentalgroup; 


2. toascertain the degree of success as- 
sociated with a developmental reading pro- 
gram when employed with an experimental 
group manifesting limited verbal skills; and 


3. to ascertain the effect of a specialized 
program of study skills, developmental 
reading and psychological counseling on 
the attrition rate of the experimental group. 


SAMPLE 


Ninety beginning freshmen students at the Univer- 
sity of Missouri-Rolla were utilized as subjects, The 
sample was sub-divided into three groups of thirty 
students. The Experimental Group represented 
freshmen students that had participated in the UMR 
Summer Pre-Registration Program, 1966, and had 
volunteered for enrollment in the one hour, no credit, 
bi-weekly study skills and developmental reading 
program. Psychological counseling was available 
to the students as was felt necessary or desirable by 
the students and their instructor. The Control Group 
I represented freshmen students that also had par- 
ticipated in the Summer Pre-Registration Program, 
1966, but were not enrolled in the study skills and 
developmental reading program nor provided coun- 
seling. The Control Group II represented freshmen 
students that had not participated in the Summer Pre- 
Registration Program, 1966, were not provided spe- 
cialized instruction, nor became involved in counseling. 


The three groups were matched in terms of their 
declared major field of study and semester hour 
course load. The sample included ceramic, chemi- 
cal, civil, electrical, mechanical, and metallurgi- 
cal engineers as well as chemistry, geology, and 
mathematics students. The ninety subjects were 
also matched on the basis of their obtained quantita- 
tive and verbal scores on the Cooperative School and 
College Ability Test (SCAT). Further, the subjects 
were matched on the basis of scores earned on the 
Missouri College English Test(MCET) and Missouri 
Mathematics Placement Test (MMPT). 


The analysis of variance (3) procedure was em- 
ployed at the beginning of the project to discern if 
any statistically significant difference existed among 
the three groups on the obtained SCAT, MCET, and 
MMPT scores as well as semester hours attempted. 
No significant difference was found at the . 05 level. 


METHODOLOGY 


Ninety subjects were selected for participation 
in the investigation at the time of pre-registration 
or at the time of fall registration. The enrollment 
of all participants in the study was verified after the 
end of the late registration period. In two cases, 
the students selected for participation in the Experi- 
mental Group and Control Group I did not arrive on 
campus to begin their class work. The criteriafor 
the selection of the two substitute students was identi- 
cal to the criteria employed in the selection of the origi- 
nal Sample. Bothcontrol groups were selected infinal 
form just after the close of the registration period. 
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Subsequent to the termination of the pre-registra- 
tion program, students selected to compose the Ex- 
perimental Group were contacted by letter and in- 
formed of the objectives of the project. Further, 
it was the intent of the investigators to provide a 
bridge for the students to assist in their transition 
from the high school academic setting to the univer- 
sity academic setting. 


The class meetings were scheduled for two hours 
weekly (T and Th) and the participants were to receive 
no academic credit. A course outline was provided 
and followed closely during the entire project. Coun- 
seling appointments were scheduled as the partici- 
pants or the instructor felt counseling was needed. 


While the minute details of the methodology will 
not be discussed, the present investigators desire 
to point out that all reading test data were collected 
during the initial phases of the project and during the 
termination phase of the project. These data were 
obtained by employing the pre-post test procedure 
with alternate forms of a diagnostic reading exami- 
nation. The basic reading materials employed inthe 
project were Miller’s Increasing Reading Efficiency 
(4) and Maintaining Reading Efficiency (5)workbooks. 
This basic approach was supplemented with various 
mechanical devices. In addition, Robinson’s Effec- 
tive Study (7) textbook provided the basic frame of 
reference for the intensified instruction in study 
skills and study habits. 


The first semester GPA and semester hours 
passed were obtained subsequent to the end of the 
first semester, This period of time was allotted to 
insure that any grade changes had taken place. Dur- 
ing the following two semesters, data relative to the 
participants’ GPA’s and attrition rates were collected 
and analyzed. 


The analysis of variance technique (3) was utilized 
to investigate the differences between mean semes- 
ter GPA’s of the three groups. Further, the same 
statistic was employed to ascertain the differences 
in mean semester hours attempted and completed 
by the experimental and both control groups. The 
critical region was established at the . 05 level. 


The ‘‘student’s’’ t statistic (6) was employed to 
discern the differences of the pretest mean reading 
scores (reading rate, comprehension, and efficiency) 
and posttest mean reading scores. The critical re- 
gion was established at the . 01 level. 


RESULTS 


The primary purpose in this study was to investi- 
gate the relationship between specialized instruction 
in study skills, developmental reading and counseling 
and academic progress. Secondary objectives in the 
study were the following: (1) to investigate the de- 
gree of success associated with the developmental 
reading procedures when utilized with a highly scien- 
tific and techologically oriented sample; and (2) to 
initiate an investigation of the attrition rate of the 
experimental and control groups. 


Contained in Table 1 are the analysis of variance 
summary tables for difference between mean semes- 
ter hours attempted and semester hours completed 
forthe Fall, 1966, Spring, 1967, and Fall, 1967 
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TABLE 1 


ANALYSIS OF VARIANCE FOR THE DIFFERENCES AMON 


SEMESTER HOURS COMPLETED 


G SEMESTER HOURS ATTEMPTED AND 


Semester Variable Source df MS F 
Fall, 1966 
Semester Hours 
Attempted 
Treatment 2 3. 82 1.82 
Error 87 2.10 
Semester Hours 
Completed 
Treatment 2 21. 24 2.43 
Error 87 8. 73 
Spring, 1967 
Semester Hours 
Attempted 
Treatment 2 12. 59 1.01 
Error 83 12, 45 
Semester Hours 
Completed 
Treatment 2 39. 97 2.16 
Error 83 18. 54 
Fall, 1967 
Semester Hours 
Attempted 
Treatment 2 .01 . 002 
Error 66 3. 36 
Semester Hours 
Completed 
Treatment 2 ‚59 
Еггог 66 20. 47 


Not Significantly different from Zero at the . 05 level 


semesters for the experimental and control groups. 


Presented in Table 2 are the analysis of vari- 
ance summary tables for the difference between mean 
Semester GPA’s for the Fall, 1966, Spring, 1967, 


and Fall, 1967, semesters for the experimental and 
control groups. 


TABLE 2 


ANALYSIS OF VARIANCE FOR THE DIFFERENCES 
AMONG OBTAINED GRADE POINT AVERAGES 


Semester Source df MS F 
Fall, 1966 
Treatment 2 1.48 3, 53* 
Еггог 87 .42 
Spring, 1967 
Treatment 2 41 „?Ч 
Еггог 83 .61 
Fall, 1967 
Treatment 2 0f 1.31 
Error 66 67 


*Significantly different from ze. 


roatthe 705 level 


Developmental Reading 


Table 3 contains the means and standard min 
tions of the experimental group for the pretest an nd 
posttest reading rates, reading aa ea а 
reading efficiency scores. Also presented are 1 
resultant ‘‘student's’’ t values from the pip quen 
analysis regarding the test of equality between р! 
test and posttest mean reading scores. 

Attrition Rate 
" ups 

Indices of the attrition rate among the thr С) es 

are contained in Table 4, The numbers are 


cessive 
the enrollment figures at the start of each suc 
semester, 


DISC USSION 


The analysis of variance (Table 1) of оед 
between mean Semester hours attempted and S dl 
ter hours completed by the experimentaland сет ari- 
£roups suggests that there was no significant er, 
ty between the groups for either variable. " vel 
this analysis provides no data regarding the le 


Ver 
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TABLE 3 
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MEANS, STANDARD DEVIATIONS, AND “STUDENT'S” t VALUES FOR TEST OF EQUALITY BETWEEN PRE- 


AND POST- READING SCORES 


Variable Pre Post нє? 
Rate in Words per Minute 
Mean 196. 79 734. 71 19. 66* 
SD 40. 81 456. 44 
Comprehension in Percentage 
Mean 80. 07 73. 96 2.07 
SD 11. 46 10. 61 
Efficiency in Words per Minute 
Mean 156. 54 514. 69 6. 60* 
SD 33. 51 284. 95 


* Significantly different from zero at the . 01 level 


TABLE 4 


NUMBERS OF STUDENTS THAT HAVE ENROLLED 
DURING THE THREE SUCCESSIVE SEMESTERS 


Experimental Control I Control II 
Fall, 1966 30 30 30 
Spring, 1967 28 29 29 
Fall, 1967 26 23 20 


at which the three groups completed the course work 
attempted, 


When the semester GPA's obtained by the three 
groups were submitted to an analysis (Table 2), a 
significant difference was discerned for the initial 
semester of the study and no significant differences 
for the final two semesters. Further analysis of the 
first semester GPA's by the Tukey (3) method for 
multiple comparisons indicated a difference between 
the Experimental and Control II groups and apparently 
accounted for the rejection of the hypothesis of mean 
equality. The fact that a significant difference in 
GPA's was discerned during the initial semester 
would suggest that the time spent in special instruc- 
tion and counseling was valuable during the initial 
semester of enrollment. Further, it may suggest 
that the experimental group progressed more rapid- 
ly with their adjustment and transition from high 
school. Also discerned from the data was that the 
experimental group maintained a constant level of 
achievement and the two control groups improved to 
a level similar to the experimental group. This was 
especially the case during the third semester of en- 
rollment when many of the students in the control 
Groups had terminated their education. In essence, 
ie Withdrawal of the lower achieving students tended 
Goa’ the groups more homogeneous in terms of 


was a important to all engineering educators 

data we attrition rate of the three groups. These 

that thi re presented in Table 4. The fact was noted 

percent Control Group II had been reduced by 33 

a 13 Не the experimental group suffered only 

will t Cent loss. Assuming these attrition rates 
епа to level off as the participants approach 


graduation, this experimental approach would help 
considerably to assist engineering students in suc- 
cessfully completing their education once initiated. 


The data presented in Table 3 strongly suggested 
that the reading phase of the experimental project 
was highly successful, both in terms of increased 
reading rate and no significant decrease in the level 
of comprehension. Further, the efficiency scores 
indicated a high positive change which implies a more 
effective and efficient reading process on the part of 
the experimental group as a result of the reading in- 
struction. Discerned was the high level of compre- 
hension at the onset of the project and the below aver- 
age reading rate. The rate was increased nearly 
300 percent without a significant loss in comprehen- 
sion level. It would appear that the below average 
verbal skills of the experimental group did not hinder 
their reading growth. The students seemed to be 
highly motivated to become involved in the reading 
program. They saw the reading skills as vitally im- 
portant, both in terms of the present academic pro- 
gram and as an avenue for keeping abreast with a 
rapidly changing technological field, 


It seems reasonable to conclude that the experi- 
mental project was successful, both in terms of GPA 
and reduced attrition rate. Further, it would appear 
that a developmental reading program can assist stu- 
dents with limited verbal abilities to progress toward 
more effective and efficient reading skills. 


FOOTNOTES 


1. R.N. Sawyer is Associate Professor of Psycho- 
logy at the University of Missouri- Rolla. 


2. L.W. Martin is Director of Institutional Studies 
at the University of Missouri-Rolla. 


REFERENCES 


1. Engineering College Administrative Council, 
American Society for Engineering Education, 
Factors Influencing Engineering Enrollment, 
October, 1965, pp.1- 5. 


2. Greenfield, L. B., ‘Attrition Among First Semes- 
ter Engineering Freshmen," Personnel and 


THE JOURNAL OF EXPERIMENTAL EDUCATION 


Guidance Journal, 42:1003- 1010, June, 1964, 5. Miller, Lyle L. , Maintaining Reading Efficiency, 


Developmental Reading Distributors, Laramie, 
1966. 


Guenther, William Cs; Analysis of Variance R 


Prentice-Hall, Inc., Englewood Cliffs, 1964, 6. 


Ostle, Bernard, Statistics in Research, Iowa 
pp. 3-63, 174. 


State University Press, Ames, 1963, pp. 119, 


528. 
Miller, Lyle L., Increasing Reading Efficiency, 7. Robinson, Francis P., Effective Study, Harper 
Holt, Rinehart, and Winston, New York, 1964. 


and Brothers, New York, 1962. 


You Are Invited to Help Launch a New Professional Quarterly 


ENVIRONMENT AL EDUCATION i 
zenry that is knowled geable concerning our bio 
problems, aware of how to help solve these р 
their solution. How better to achieve this perv. 
of this professional Journal, 


roblems, and motivated to work toward 
asive ecological conscience is the focus 


i communications, for conservation educators, for 
What This Journal Is environmental designers, for interpretive natu- 
cation is a new quarterly de- ralists, for regional planners, for resource in- 
voted to discovery and dissemination in the vestipators „апі managers, for Practitioners of 
emerging field of multidisciplinary conservation Public relations in resource industries, for KCN 
communications, Through a recording of insight — Vation information specialists, for outdoor writers, 
and information about pioneer research and inter. fOr lay leaders in land and water bake oa 
Pretation projects and programs, this Journal will for | al А {е SE abd : ч, "pes d Anc 
act as a catalyst in encouraging and exchanging applying seite Ways 0 SE oping a р е 3 fan- 
effective studies and developments toward broad servation conscience” on the part of America's 
public ecological awareness, understanding, апд “itizenry. 
action, 


Where 


Whom It's For Environmental Education is published four times 
This is a professional Journal for all those who 2 year by Dembar Educational Research Services, 
are beginning systematically to study and prac. Inc., Madison, Wisconsin, the publishers of The 
tice the new interpretation of America's resources Journal of Educational Research and The Journal 
—their essential integrity, their inexorable rela. of Experimental Education, It will initially be | 
tionships with modern man, their beauty, their 32 Pages, 7x10; priced at $2 a copy, $7.50 a 
bio-physics, their utility, their socioeconomics, their year, $12 for two years; student rates will be 
conservation: for researchers in education and $4.75 a year, 


Inscribe Your Name Today 


DEMBAR EDUCATIONAL RESEARCH SERVICES, 


- — | want to bea sponsor. Plea 
two (2) years at $12. 


— D fie to try Environmental Education for а year. Please enter my subscription at 
7.50. 


INC., BOX 1605, MADISON, WISCONSIN 53701 


se enter my subscription to Environmental Education for 


I'm interested in Environmental Education. Please send me a copy of the current ¢ 
issue for $2. 


—. I enclose a manuscript for your consideration. 


— lendose a check, __ Bill me. Bill am T ы ш, - = 
МАМЕ ee —— PHÓ - n 
ADDRESS _ Es - RR — ERI " — 
i j се ES = - ———_. STATE 


THE JOURNAL OF EXPERIMENTAL EDUCATION 
(Volume 37, Number 4, Summer 1969) 


CONFIDENCE LEVEL AND SIGNIFICANCE LEVEL: 
SEMANTIC CONFUSION OR LOGICAL FALLACY 
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ABSTRACT 


Despite repeated warnings against its usage, researchers and textbook authors continue to employ the 
phrase ““1еуе1 of confidence’’ as a synonym for the phrase “level of significance.” Three possible sources of 
this incorrect usage are noted: (a) imitation; (b) the related but different concepts of fiducial inference and 
confidence intervalestimation; and (c) the myriadof terms associated with confidence. It is recommended that 
the magnitude of level of confidence be restricted to that of the complement of the level of significance and also 
that the term level of confidence should be used only in connection with interval estimation. 


MOSTELLER, Rourke, and Thomas, ina text 
largely based upon materialused on the popular NBC 
“Continental Classroom"! course in probability and 
statistics, have made the following distinction be- 
tween level of confidence and level of significance. 


In a testing probiem, we speak of the 
significance level of the test. It measures 
the probability of rejecting the null hypothe- 
sis when it is true. It is the complement of 
the confidence level, if a confidence level is 
used to make the test. Thus, if the confidence 
level is chosen as 0. 95, the significance 
level is 0.05 (16:304). 


Somewhat differently, McNemar has cautioned 
that “the term ‘levelof confidence’ should not, as 
it frequently is, be misused in place of ‘level of sig- 
nificance.’ The first term pertains to interval es- 
timation, the other to hypotheses testing (15:93). ” 


Consequently, while reading a recent elementary 
Statistics textbook (14), we were jarred by the re- 
peated use of the phrase, ‘‘5[ or a percent level 
5] Confidence’’--when the author explicitly was refer- 

B to the critical region. Clearly, level of confi- 
гс Was being used as a synonymous expression 
Vel of significance. 


eed Sensitized to this questionable equating of 
evel o lons, we began to discover the ‘‘5 percent 
Croppin, Confidence’’--or 10 percent ог 1 percent-- 
Places £upin quite varied and sometimes unexpected 
another jn article in Educational Horizons (13:103), 

in the Journal of Educational Measurement 


зь 


(20:111), anda handfuloftexts, uncovered by a limit- 
ed search of the literature, provided further exam- 
ples of this usage. Apparently, the use of the 5 per- 
cent level of confidence is unrelated to prestige; it 
is employed by Edwards (6:242) and by Guilford (9: 
152). Moreover, date of publication does not seem 
to Бе а factor in the use of this terminology (e.g. , 
11:26 and 12:13). Hence the equating of level of con- 
fidence with level of significance appears to be wide- 
spread and of long standing. 


Although the confusion of levelof significance 
withlevelof confidence has been noted before (e.g. , 
3 and 18), there has been little speculation about the 
origins of that confusion. Three explanations seem 
plausible. It may be that statisticians, like everyone 
else, learn by imitating exemplars. Hence, the first 
explanation, and perhapsthe proximate cause, is the 
equivalent use of the two terms in some textbooks 
and journals--and even in a dictionary. 


For example, consider this definition from a dic- 
tionary of psychological terms. 


Confidence level specifies the percent- 
age of samples that may be expected to fall 
outside the confidence interval: one speaks 
of a statistic as stable or significant at the 
5 percent or 1 percent level, Although this 
meaning is rigorously definable, it is confus- 
ing; 5 percentis not the percent in which we 
are confident; it is the percent that we must 
risk (7:206). 
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Suggestively, much of the English and English (7) 
discussion of confidence is found under the rubrics 
of fiducial limits and fiducial probability. And here 
we arrive at the most likely explanation for the con- 
fidence level vs. level of significance confusion-- 
namely, the related, but different concepts of fidu- 
cial inference and confidence interval estimation, 


Although we donot intend to reopen the once acri- 
moniously debated--and yet unsettled--question con- 
cerning the relationships between fiducial inference 
and confidence interval estimation, it might prove 
worthwhile to revisit those concepts, First, what 
is meant by fiducial inference? Ina statistical dic- 
tionary the following definition is offered: 


Fiducial Inference: A type of statistical 
inference based on fiducial distribution intro- 
duced by R. A. Fisher. The object of the in- 
ference is to make probabilistic statements 
about the values of unknown parameters and 
to that extent it resembles the theory of con- 
fidence intervals. In simple cases results 
Írom fiducial theory agree with those from 
the theory of confidence intervals, but this 
is not so in general When this type of in- 
ference and the related confidence-interval 
type were first propounded, they were often 
confused. The confusion survives to the ex- 
tent that “fiducial” is sometimes applied toin- 
ference of the confidence-interval type (10:108). 


It is doubtful whether this short definitioncan illum- 
inate the murky concept of fiducial inference. How- 
ever, these two definitions do reflect the historical 


confusion of fiducial and confidence interval inferences, 


Now that we have touched on the elusive notion of 
fiducial inference, let us briefly reexamine some 
statistical competitors: confidence, confidence inter- 
vals, and confidence limits, After all, most applied 
Statistics textbooks do pay homage to the Neyman- 
Pearson concept of confidence. An excellent defini- 
tion of confidence is provided by Chandler: 


Confidence, a concept customarily re- 
served for discussions of interval estimation, 
is the faith which one is willing to place in a 
statement that an interval established bya 
sampling process actually contains or bounds 
a parameter of interest. One generally ex- 
presses this faith statistically by affixing to 
each intervala confidence coefficient, or con- 
fidence probability, which can be written as 
1-€, where € =p/100for 0< p< 100 апар 


is usually taken to be a very small number 
(3:429). 


As is wellknown, the confidence coefficient is not 
the usual relative frequency probability. The proba- 
bility of a given confidence interval containing the 
Parameter under consideration is either zero ог one, 
The confidence coefficient does represent a degree 
of faith in the particular procedure for constructing 


confidence intervals, that over a large number of 
trials with equal si 


However, the meaning and interpretation of con- 
fidence apparently areconcepts under which the logi- 
cal underpinnings are not as strong as is usually im- 
plied by applied statisticstexts. A recentbookreview 
in Science underscored this point: 


The deeper difficulties of the theory 
[Neyman- Pearson confidence intervalesti- 
mation] appear when it is recognized that 
the interpretation of individual sets of data 
is after all a necessity. A theory which 
deals only in long-run frequencies does not 
possess in itself the resources to deal with 
such interpretation (5:663). 


None the less, most of our textbooks in applied 
statistics reflect the historical legacy of both the 
Fisherian and Neyman-Pearson schools. In these 
texts significance tests and null hypothesis lie com- 
fortably along side the two types of errors and con- 
fidence. Moreover, even most authors of applied 
statistics texts admit that the statistical logic of fidu- 
cial probability and confidence differs. But with rare 
exception (e. g., 19:113) these same authors dismiss 
fiducial limits and fiducial probability (if mentioned 
at all) with a footnote such as: ''Strictly speaking, 
these are ‘fiducial limits’ and not confidence limits. 
For practical purposes the distinction is so slight as 
to be not worth making, and because of its greater 
intelligibility the term confidence limit is used here 
(2:34).’’ Torecapitulate, it has been shown that fidu- 
cial inference has been confused with confidence in- 
tervalinference. It shouldbe added that Fisher (e.g. 5 
8:198) used the term fiducial probability to refer to 
the probabilities associated with the area between 
fiducial limits and with the area in the tails of the 
fiducial distribution, i, e., the level of significance. 
Because Fisher employed fiducial probability in two 
complementary ways, latter writers have used level 
of significance with the same two meanings. dor ex- 
ample, Rozeboom (17:423) speaks of a ‘‘99 © level 
of significance” in discussing an acceptance region. 
By contrast, in customary usage the region of rejec- 
tion, not the region of acceptance or non- rejection, 
is referred to in terms of level of significance pi 
for example, 1:34). Given these connections, cO а 
dence interval inference with fiducial inference an 
level of significance with fiducial probability, s 
not surprising to find levelof confidence being used d 
onymously with level of significance- -whether re cod 
ring to the region of rejection or to the region of a 
ceptance or non- rejection. 


The third probable cause (from a semantic oe 
Point) of the confusion between level of confide) f 
and level of significance is the bewildering Sude 
terms associated withconfidence. Examples dirus co- 
confidence interval, confidence level, conf ident nits. 
efficient, degree of confidence, and confidence ит 


ible 
To summarize, we have suggested three pedi 

causes for the confusion between level of significa) he 

and confidence level, These causes are: P. own 

imitation of the terminology employed by well- fo de- 
applied staticians, (b) the historically SERERE ЙЕ 
velopment of Fisherian and Neyman- Pearson Scally 

tics involving the deceptively similar but ЮЕ enc 

different concepts of fiducial inference and cor terms 

interval estimation, and (c) the plethora О! 


tes 
EA 
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associated with the concept of confidence. Obviously, 
these causes are not mutually exclusive. 


It would appear, then, that in most applied text- 
books seldom is a logical fallacy committed in refer- 
ring to levelof confidence or to level of significance. 
Most authorities realize that, by one convention, а 
5 percent level of confidence can refer to the proba- 
bility of committing a Type I error. It is unhappily 
true that differing conventions (what we have called 
semantic confusion) can mislead the student, perhaps 
to the point of committing logical fallacies. 


It is unfortunate, especially in view of the logical 
haze surrounding both confidence and fiducial proba- 
bility, that applied staticians are so inconsistent in 
their choice of terminology relative to level of signifi- 
cance or level of confidence. Because commonly 
accepted conventions do have their place in the de- 
velopment of a discipline, we support and advocate 
the convention given by Mosteller and others (16) at 
the beginning of this paper, that is, that we restrict 
the magnitude of confidence level to that of the com- 
plement of the significance level. Indeed, this is the 
convention sanctioned by at least one of the original 
proponents of confidence theory (4). Furthermore, 
for tests of significance the experimenter is concerned 
with two types oferror, i.e., (a) the probability of 
rejecting the null hypothesis when it is true and (b) 
the probability of failing to reject the null hypothesis 
whenitis false. By contrast the level of confidence 
is a sufficient description of the potential accuracy 
in interval estimation. Thus although the level of 
significance is formally the complement of level of 
confidence there also is a difference in the information 
conveyed (see 3:429- 430). Therefore, we must urge 
with MeNemar (15) that level of confidence should be 
restricted to interval estimation while level of signifi- 
cance should refer to hypothesis testing. 
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TIME-COMPRESSED RECORDINGS 


THOMAS G. STICHT 
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ABSTRACT 


Time-compressed, tape recorded messages were used to deter 
the same amount of time required to listen to the uncompressed message once, would improve listening com- И 
ргеһепѕіоп ѕсогеѕ оѓ high and low aptitude men, The results indicated that for both groups of subjects, listen 


ing to the compressed messages twice improved comprehension scores over scores obtained by listening to the 
compressed tapes once, Comprehension of repe: 


obtained by a single listening to the uncompress 


THE RATE of Speech of a tape recorded mes- 


relics, taken from Form 1A of the Sequential Tests 
Sage can be accelerated by an electromechanical of Educational Progress, was used in the present 
Process. This process involves the periodic dele- Study. The tape recorded listening selection was 
tion of small temporal segments of the Speech, while processed? to produce compression ratios of 0, 36, 
the remaining speech segments are connected to pro- 46, 53, and 59 percents. А 20-item ‘fill-in-the- _ 
duce connected discourse (cf. , 3, tor a more com- blank” test was prepared to evaluate listening com 
lete description of this process). Through this dele- 


prehension. The test questions required the recall 
of factual information contained in the listening um 
lections. Tests were scored by counting the num 

of correct responses, 


tion and sampling technique, the time required to 
present the message is decreased. Hence 5 uch 
Speech is sometimes called time compressed Speech. 


If a message is time-compressed by 50 percent, i 

it is possible to present the Compressed version Subjects 
twice, in the same amount of time required to pre- Eighty Army inductees served as 55, Fortv 55 
sent the uncompressed version Once. Fairbanks, had Armed Forces Qualification Test (AF QT) 
and others (1) compared the comprehension of mate- scores less than 30 and formed a low mental apti- 
rial compressed by 50 percent, but presented twice, tude (LMA) grou The remaining forty men had 
with the comprehension of the uncompressed mate- AFQT adores of n Or above and constituted a high 
rial requiring the same amount of time for presen- mental aptitude (HMA) i 

tation. They found a slight, but not statistically sig- P Broun. 
nificant, improvement in comprehension with the re- 


Peated, time-compressed message. The double Procedure 
presentation appeared slightly more successful with sub- 
men of moderate than of high mental aptitude. This , The forty men in each aptitude group ee ten 
Suggested that the repetition procedure might prove divided into four groups of ten. Each group 0 entá” 
more successful with very low aptitude men whose listened to a different pattern of repeated pres 115- 
ability to learn by listening was being studied (4). tions of the listening selection. One Patent avi 
Hence, several repetition procedures were used to tened to the passage compressed by 36 per ent GOH 
determine if the learning performance of high and then the Passage was repeated with 59 perc rcent 
low aptitude men could be improved over that for a Pression. A second subgroup heard the 59 cent 
Single presentation of uncompressed material, Compressed version first, and then the ga Pi to 
compressed version, A third subgroup listen ion 
METHOD the passage presented at 46 percent compres a 
followed by the same passage presented at 5 Га to 
Materials cent compression, The fourth subgroup listene iie 
| the same versions as the third group, but with t 
A selection on the use of Carbon 14 for dating g ; 


order of compression ratios reversed. 
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PERCENT COMPRESSION 


FIGURE 1 


COMPREHENSION SCORES FOR HIGH AND LOW APTITUDE Ss FOR SINGLE (FILLED SYMBOLS) AND 
DOUBLE (UNFILLED SYMBOLS) PRESENTATION OF LISTENING SELECTION (SEE TEXT FOR 


EXPLANATION OF ABSCISSA). 


The total listening time needed to listen to the 
selection once at 36 percent compression, and then 
again at 59 percent compression, came to 105 per- 
cent of the time required to listen to the uncom- 
pressed selection once. To listen to both the 46 
and 53 percent compressed versions, the total lis- 
tening time was 101 percent of the time necessary to 
present the uncompressed message. Listening time 
for the latter was 2 minutes, 35 seconds. 


The recordings were presented to each S individ- 
ually by means of a Wollensak tape recorder. Test- 
ing was accomplished in a sound-deadened room. 
Subjects listened to both versions of the repeated 
message before taking the comprehension test. The 
comprehension score was the percent of items cor- 
rectly answered on the test. 


RESULTS AND DISCUSSION 


The results are presented as the open characters 


in Figure 1. In this figure, the bottom abscissa 
presents the compression ratios, while the top ab- 
scissa gives the corresponding speech rates in wpm. 
In each instance, the upper digits on the abscissa 
indicate the compressed version presented first. 
For comparative purposes, the filled symbols are 
presented, These symbols show how Ss similar to 
those of the present study scored on the same listen- 
ing selection when it was presented only one time at 
the compression ratio indicated by the upper figure 
along the abscissa. These data were obtained from 
Ss tested in previous studies (4) who were matched 
as near as possible to the Ss of the present study 
with respect to AFQT score. Ten Ss were included 
in each of the mental aptitude subgroups. 


The dotted lines in Figure 1 are reference lines 
showing the comprehension level for the single pre- 
sentation of the uncompressed passage for the LMA 
and HMA Ss. Improvement due to listening to re- 
peated compressed passages would be indicated by 


62 THE JOURNAL OF EXPERIMENTAL EDUCATION 


Scores above the reference line. Three aspects of 
these data are notable, First, listening to the se- 
lection twice, with the various combination of com- 
pressed material, did not improve performance over 
that obtained with one presentation of the uncom- 
pressed story. This was true for both HMA and 
LMA groups. Second, the Sequence of presentation 
did not differentially affect performance. The only 
suggestion that the sequence of presentation may 
have improved performance is with the LMA group 
and the 59 percent - 36 percent sequence, However, 
the score for this sequence is not significantly higher 
than that for the single presentation nor the presen- 
tation of the 36 percent - 59 Percent sequence 
(Fisher exact probability test), 


A final feature of interest in the data of Figure 1 
is that repeating the selection with the fast w ord 


curves for the single present- 
The fact that there is a notable decrease in 
comprehension with single presentations of word 
rates of 325 or 375 wpm is well documented for a 


As mentioned earlier, Fairbanks, and others 


(1) reported a very slight improvement in compre- 
hension with double p 


material over that ob 
tion of uncompressed material, 


listening materials through the simple repetit 
materials, with men of 


More research is needed along these 


1. 


2, 


1. 


- Fairbanks, Gog Guttman, N. 


- Sticht, T, G, 


FOOTNOTES 


eported in this paper was per- 
i o илана Division No. 3 (Recruit 

Training), Presidio of Monterey, California, 
under Department of Army contract with the 
George Washington University. The contents 
of this paper do not necessarily represent 
the official opinion of the Department of 
the Army. 


The speech compression equipment was the 
Eltro Information Rate Changer, distributed 
by Gotham Audio Corp., 2 West 46th Street 
New York, 10036. The equipment is iden- 
tified for purposes of documentation and 
does not imply endorsement by either Human 


Resources Research Office or Department 
of Army. 
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ABSTRACT 


The peer nomination technique was investigated as a predictor of academic success among members ofa 
college freshman class. The technique provided scores which were reliable and which were valid against a 
grade point ratio criterion. The peer nomination scores were shown to tap variance independent of SAT scores 
and, therefore, their applicability was suggested in academic counseling as a predictor which can be used to 


identify students who may face academic difficulties, 


THE USE of the peer nomination technique as a 
predictive device in certain practical settings is be- 
coming accepted as psychometrically sound. Mili- 
tary applications have involved predicting a variety 
of variables including: leadershipqualities (2), pre- 
flight training failure (4), and promise as future of- 
ficers (6), In industry the technique has been used 
to predict supervisory and managerial success (7, 5). 


One of the intriguing aspects of the peer nomina- 
tion technique is its reported success in achieving 
high reliability and respectable validity in groups that 
have been together for only relatively short periods 
of time. Hollander (2) achieved good results in a 
Naval OCS situation when the cadets had been together 
less than one week and had not even begun their regu- 
lar training. Even better results were obtained af- 
ter the groups had been together for only a few weeks. 
More recently Hollander (3) has reported a follow- 
up of his earlier work suggesting that the predictive 
validity of peer nomination scores hold up well 
against distant criteria. 


The present investigation was designed to deter- 
mine the efficacy of the peer nomination technique 
in a collegiate setting for predicting academic suc- 
cess among newly arrived freshmen. 


METHOD 


On each of three occasions, peer nominationforms 
Calling for selections on the basis of forecasts of 
Srades and perceptions of academic motivation were 
administered to 112 freshmen women and 140 fresh- 
men men organized into nineteen dormitory units. 
The first occasion (time one) was at the endof 

Orientation week” before the subjects had attended 
any regular college classes. The second occasion 
time two) was after 5 weeks of classes but before 


any grades had been officially released by the college, 
The third occasion (time three) was after 11 weeks 
of classes and after ‘‘midtermgrades" had been re- 
leased for advisory purposes. 


The forms were administered by upper-class stu- 
dents who served as dormitory counselors in the 
various dormitory units. The counselors were fully 
briefed, inadvance, in order to secure a good level 
of cooperation and to assure uniformity of conditions, 
They were instructed, however, to give only minimal 
explanation, but to assure the subjects in their units 
that the results would be kept confidential and used 


for research purposes only. The forms carried the 
statement: 


This is a study designedto investigate person- 
al factors in college success, The information 
requested is for research purposes onl: and will 
be held in the strictest confidence. In fairness to 
yourself and others and to safeguard the success 
of this study you are asked to keep the ratings 
confidential, Please avoid discussing the ratings 
you have made with anyone. 


The subjects were asked to nominate in order the 
members of their dormitory unit whom they expected 
to be the three highest and three lowest for each of 


the two variables: grade point ratio (GPR) and aca- 
demic motivation. 


Scoring followed the pattern used in other published 
studies (cf., 2). This procedure involved a direct 
weighting method to obtain peer nomination Scores, 
Each nomination in the highest position was Scored as 
*3, each in the next highest position was scored as 
+2, and each in the third highest position was scored 
as +1; each nomination in the lowest position was 
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S -3, each in the next lowest position was 
ec] ds -2, and each in the third lowest position 
was scored as -1. An algebraic sum was obtained 
of the weighted nominations receivedby each subject 
and divided by the n of the group. A linear trans- 
formation was then computed to permit the convenience 
of working with a 2-digit score withoutan intervening 
decimal. 


RESULTS 


Table 1 presents the intercorrelations among the 


TABLE 1 


INTERCORRELATIONS A 
FERENT TIMES 


two peer nomination variables administered on и 
Occasions. It is evident that the forms ши дат 
reasonably stable results over the 11-week рте 
and that there was substantial overlap between ha 
two peer nomination variables. Table 2 show T 
validities of the peer nomination scores and of cr 
Scores against earned GPR criteria. All py 
efficients were statistically significant and some w jh 
sufficiently large to have practical significance. 


A AL геа substan- 
expected, the peer nomination scores showed sub 


i idities aS 
tial validity even at time one, but higher valise 
a function of the length of time the groups ha 


MONG PEER NOMINATION SCORES FOR FORMS ADMINISTERED AT THREE DIF- 


Form and Time 2 


3 4 5 s 
1 Grade Point Ratio--time 1 .70 254 86 67 2 
2 Grade Point Ratio--time 2 .72 . 62 86 «#8 
3 Grade Point Ratio--time 3 .48 17 att 
4 Motivation--time 1 + 65 39 
5 Motivation--time 2 i | 


6 Motivation--time 3 


Note--All intercorrelations are significant at the ‚01 level, 


TABLE 2 


VALIDITY OF PEER NOMINATION SCORES AND 


5 
SAT SCORES AGAINST SEMESTER GRADE POINT RATIO 


GPR Criteria 
Predictors Ist Semester 2 


Wine Ix eee 
nd Semester 3rd Semester 4th Sem 


Grade Point Ratio 


22 
.34 .28 98 с 
Motivation .28 n 20 AT 
Time 2 
Grade Point Ratio .39 .35 30 м 
Motivation .50 .45 40 B5 
Time 3 38 
Grade Point Ratio .58 .42 42 
Motivation . 67 47 42 e 
SAT 
Verbal .31 33 31 " 
.24 
Math .35 ‚28 .30 


Note--All validities are Significant at the -01 level, 


TITUS 


together. It is notable, too, that reasonably go od 
validities were obtained against earned GPRs in later 
£rading periods, This, of course, must be under- 
stood as largely a function of the high intercorrela- 
tion among semester GPRs (mean intercorrelation 

over four semesters was . 67). 


DISCUSSION 


The primary value to be gained from a prediction 
of academic success among persons already selected 
and admitted to an academic institution is to identi- 
fy as early as possible students needing special at- 
tention. In institutions which are reasonably selec- 
tive, one might expect, other things being equal, 
thatall (or at least the vast majority) of the selectees 
will succeed. But clearly such is rarely the case 
because selection is usually on the basis of ability- 
achievement factors, whereas it is very often the 
case that many other factors play a significant role 
in college success. 


The method of peer nomination offers the possi- 
bility of utilizing students’ perceptions of one another 
on subtle and often elusive variables (e. g., academic 
motivation). The data presented here suggest that 
the technique is sufficiently reliable and valid to be 
of interest. But the question remains: Does it con- 
tribute new measurement variance beyond that which 
was available at the time when the students were 
selected? Partialcorrelations were computed between 
peer nomination scores at time two? and first- 
semester grades with SAT verbal and SAT math 
scores held constant. The partial correlation for 
peer nominations of GPR with earned GPR was . 34; 
for peer nominations of academic motivation with 
earned GPR it was . 47. It appears that the nomina- 
tion technique ‘‘tapped’’ variance not accounted for 
by the SAT scores. A multiple correlation of . 56 
was obtained for the two peer nomination variables 
(at time two) and the two SAT scores against first 
semester earned GPR. Beta weights for the vari- 
ables were as follows: peer nominations for GPR 
. 03, peer nominations for academic motivation .35, 
SAT verbal .19, and SAT math. 15. Thus the peer 
nominations for motivation contributed the most to 
the prediction of the earned GPR criterion. This 
finding is further born out ina group-by-groups anal- 
ysis of the data. Theacademic motivation variable 
at time two yielded validities from a low of . 37 to a 
highof. 78 (meanr =, 57)4 against earned first semes- 
ter GPRs, Inallbut two of the dormitory units these 
validities were statistically significant; at time three 
the validities were statistically significant in all the 
dormitory units. Furthermore, the group-by-group 
analy sis showed the peer nomination scores for moti- 
vation to be substantially more consistent in their 
validity than the SAT measures. The SAT verbal 
showed a range of validity coefficients from -. 09 to 
.81 (mean r=.41). The SAT math showed a range 
of validities from -. 29 to. 55 (mean r -.32). Itis 
likely that the relatively good results with the peer 
nomination motivation variable reflected the students’ 
understanding of the implications of academic moti- 
vation, Thatis, as something separate from ability 
and achievement--an enthusiasm, a seriousness of 
purpose, an air of congruence with the aims and pur- 
Poses of anacademic institution. These data do not, 
of course, identify what their conception of academic 
motivation was but only that was relatively independ- 
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ent of the SAT kind of variance (this interpretation 
was further strengthened by results of factor analyses 
not reported here). 


Rating one’s classmates is difficult and, for some, 
a trying task. It was suspected that attitude toward 
the process might materially affect the accuracy of 
predictions and that there might be sex differences 
even overriding this additudinalfactor. With regard 
to the latter consideration, there were no significant 
differences between men and women in either the re- 
liability or validity of their peer nominations. It is 
perhaps notable that the women at time one and time 
two expressed slightly more resistance to the pro- 
cess of making interpersonal judgments than did the 
men. A very few subjects of both sexes declined to 
make any nominations at all, but the vast majority 
even of those who were reluctant and/or skeptical 
performed as requested. 


A section of the cover sheet accompanying the 
peer nomination forms offeredthe subjects an oppor- 
tunity to make comments. Mostofthe subjects made 
no comments at all or made comments that were 
judgedto be neutral. However, 37 percent of the 
subjects made comments at time one that were skepti- 
calor negative compared with only 8 percent at time 
two and 3 percent at time three. Examination of these 
data on a group-by-group basis showed that the groups 
which produced the highest frequencies of negative 
reactions were indistinguishable from the remaining 
groups with respect to reliability and validity of the 
peer nomination scores, 


SUMMARY AND CONCLUSION 


The peer nomination technique was applied at three 
times during their first semester at college to 252 
freshmen organized into nineteen dormitory units. 
Scores based on their nominations concerning GPR 
and academic motivation showed high reliability and 
promising validity. Since the peer nominations were 
seen to be measuring variance independent of the 
ability-achievement factors usually considered in 
academic predictions, it appears that the technique 
can provide unique information and perhaps can con- 
tribute to the early detection of students who are in 
need of special academic counseling. 


FOOTNOTES 


l. The author wishes to express his appreciation to 
Miss Patricia Siller who provided valuable as- 
sistance in the pilot work preceding this study. 


2. Time two was chosen because it appeared to offer 
the best compromise for practical applications 
betweenthe need to have information early and 
the need to have it as accurate as possible. 


3. Allanalyses based on individual dormitory units 
were limited to the twelve groups with n= 12, 


4. Mean r's reported in this paper are based on 
Fisher's (1) z transformation. 
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ABSTRACT 


A controlled experiment was conducted in forty-four fifth-grade classrooms to investigate the extent to 
which creativity and problem-solving skills of children could be nurtured through a series of self-instructional 
programmed lessons. Also studied was the relationship of such skills to (1) learner characteristics (IQ and 
sex) and (2) classroom “environment” as it may have facilitated creative thinking. Significant differences 
were found favoring the experimental group (which received the programmed lessons) Improvement in produc- 
tive thinking skills was found for both boys and girls of both higher and lower IQ. Greater gains were found in 
classrooms providing relatively little support and encouragement for productive thinking. Also the performance 
of girls on the productive thinking measures exceeded that of boys and there was a strong positive relationship 


to IQ. 


CREATIVITY in school children-- particularly 
gifted children--has been studied extensively (8, 
10), This research has led to the organization of 
programs for the gifted which include education for 
Creativity (7, 9). There is now increasing interest 
in educational programs which seek to develop the 
Creative potential of a much wider segment of the 
school population. One set of instructional materials 
which attempt to do this is The Productive Thinkin; 
Program, Series One: General Problem Solving (3). 
Developed by researchers at the University of Cali- 
fornia at Berkeley, this program consists of a set 
of programmed lessons intended to develop and 
strengthen productive thinking skills in fifth- and 
sixth-grade students. In previous studies, these 
materials produced a substantial increase in the 
creative thinking performance of students at many 
different levels of intellectual ability (2, 4). 


The major purpose of this study was to investigate 
the extent to whichthe creativity and problem-solving 
skills of fifth-grade children could be improved 
through use of such a set of programmed lessons. 
In this context, several specific aims were incorpo- 
rated: 


(1) To investigate further the extentto which pro- 
ductive thinking (i. e., creative thinking and prob- 
lem solving) abilities can be taught to elementary 


school children independent of any specific subject 
field. 

(2) To discover whether some of these skills or 
behaviors are more readily taught than others. 
For example, one aspect of problem solving is the 
asking of appropriate questions. Can students be 

taught this behavior more (or less) successfully 

than they can some other skill, suchas generating 

a set of possible solutions to a problem? 


(3) Toinvestigatethe relationshipbetween certain 
learner characteristics (intelligence and sex) and 
performance on a broad diversity of productive 
thinking measures. It is generally reported that 
girls perform slightly better than boys on most 
measures of academic achievement, Is this also 
true of measures of productive thinking? It has 
also been reported (14) that intelligence and ‘‘crea- 
Чуну” are relatively independent. Does this in- 
dependence persist throughout several different 
aspects of productive thinking and on several dif- 
ferent types of problems? In addition, is there a 
relation between intelligence or sex and the ability 
to learn productive thinking skills? 


(4) Toinvestigate the extent to whichoverall class- 
room''Environment'' facilitates productive think- 
ing in students, and the extent to which students 
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in different types of classroom environments pro- 
fit from instruction in productive thinking. The 
investigators felt that the kind of classroom atmos- 
phere (provided primarily by the teacher in her 
conduct of class discussions) might influence the 
kind and amount of productive thinking on the part 
of students in that classroom, Students іп a class- 
room where the teacher encourages а free and 
open exchange of ideas might tend to exhibit more 
of what is here considered productive thinking be- 
havior than would students in a classroom where 
the teacher maintains close control over the ideas 
which may be pursued in discussions. To what 
extent does merely providing an encouraging envi- 
ronment promote creative thinking? Are instruc- 


tional materials more (or less) effective in such 
environments? 


a totalof 704 students. (Selection was made aft 
experiment had been completed but before any 
of tests was begun. ) 


sample 
Т stimate of error, even 
with the use of classroom means (rather than individ- 


it of analysis, 


that existed in the various classrooms. The final, 
single Environment rating assigned to each class was 
a composite score derived from the several objec- 
tiveratingsthe supervisor made. Thoseratings were 
designed to indicate the extent to which the ер 
allowed free and open discussion, the degree to whic 
the teacher avoided making value judgments about И 
unusual ideas offeredby students during class 1500 
Sion, andother characteristics related to the exten 
to whichthe classroom environment could be thought 
of as facilitating or inhibiting the kinds of behavior 
considered indicative of productive thinking. The 
twenty-two classrooms with the highest ratings were 
designed as Facilitative Environment classrooms; 
likewise, those twenty-two classrooms whose com- 
Posite rating was below the median were classified 
as Nonfacilitative Environment groups. к 


Within each of these Environment blocks, half the 
classrooms were randomly assigned to the Treatment 
Group (which would use The Productive Thinking Pro- 


gram) andthe remaining half constituted the Control 
Group. 


Instructional Materials 
etna! Materials 
The Productive 


instructional m; 
the student's ab; 


Thinking Program (3) is a set of 
aterials which undertakes to develop 
ility to use important skills and strat- 
ng and problem Solving, seeks to en- 
hance his understanding of the processes of his own 
thinking, andaims to promote favorable attitudes and 
motivations toward activities which involve the use 
of the mind. Series One of this Program, which 
deals with General Problem Solving, served as the 
instructional materials in this study, Í 


The Skills taught by Series One of The Productive 
Thinkin, 


Ч Ргоргат аге those thought to be essential 
for the kind of complex and ori 


a gs, and to achieve his 
ms. The program also shows 
ceed ina systematic way when 
how to clarify a problem, and 
ant facts and conditions of the 
hisideas, Inaddition, it seeks 
tivations and attitudes required 
ance in productive thinking tasks. 


e he Productive Thinking Program 
Consists of sixteen propra — Dking Progra 
hii 


en programmed lessons, each of 


the student how to proi 
attacking a problem, 
how to attend to reley; 
problem in evaluating 
to promote those mo 
for effective perform: 


The 16-lesso, 
room hours of in 
the lessons wer 
of each week 
Was used for 
dent worked i; 


П series requires a total of 16 eae 
struction time. In the present stu T 
е administered one per day for 4 aya 
;the remaining school day in each we iur 
makeup sessions, Generally, each 3 
ndividually at his own pace and had 
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assignment to which he turned when he had completed 
the day's lesson. 


Since it was the intention of this study that the 

teacher's role in the use of these materials be held 

o a minimum, the teachers were instructed only to 

distribute the lessons and answer procedural ques- 
tions. They were not to conductclass discussions 

based on the materials or otherwise participate in the 

way they were used. (One exception to this policy was 
permitted inclasses witha number of poor readers. 
In these classes, teachers often read portions of the 
lessons aloud with small groups, or otherwise assisted 
students who had substantial reading problems.) 


Measuring Instruments and Scoring Procedures 


Most of the measuring instruments used in this stu- 
dy were developed by the Creative Thinking Project at 
Berkeley anddiffer from traditional educational tests 
in both content and structure. 2 А discussion of their 
underlying rationale may be found in Covington (1). 


Briefly, the instruments аге based onthe proposi- 
tionthat creative thinking can be conceptualized as an 
intricate and highly complex problem solving process 
which requires the individual to use and to coordin- 
ate a number of different cognitive skills as he seeks 
toreachhisgoal. Thiscomplex problem-solving pro 
cess involves: (1) the use of a number and variety 
of cognitive skills; (2) the effective management and 
deployment of these cognitive skills by what might 
be called a ‘‘master”’ thinking skill; and (3) а high 
degree of involvement on the part of the thinker--the 
individualis highly engrossed with the creative task 
and finds it meaningful and engaging. 


TABLE 1 
CLASSIFICATION OF THE T 
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Hence the measuring instruments used in this study 
attempt to present the student with thinking tasks 
which he willfind meaningful and engaging, and which 
require him to use and to coordinate a number of dif- 
ferent cognitive skills. These tasks do not involve 
the use of specialized knowledge or technical skills, 
nor do they require the mastery of any particular 
curriculum material. 


A complete list and classification of the tests of 
productive thinking used in this study is presented 
in Table 1. In this classification, the terms ‘‘con- 
vergent”’ and “divergent” thinking are used in the 
traditional way to denote thinking which, on the one 
hand, involves processing information so as to ‘‘con- 
verge” upona single solution toaproblemor, onthe 
other hand, involves the use of informationas a point 
of departure from which one’s thinking ‘‘diverges’’in- 
to a variety of new mental products. ‘‘Complex Ex- 
tended Problems” in the classification are those which 
feature a gradual development ofthe problem and re- 
quire the student to use a number of different cognitive 
skills as he proceeds. The student responds at sev- 
eral points and is presented with feedback in theform 
of more information about the problem after each 
response opportunity. Those measures labeled''Brief 
Problems” require the use of relatively few cognitive 
strategies and do not employ response-feedback se- 
quences; instead, they present the entire problem to 
the student at once, and he then makes his response(s). 


All the problems are further classified as verbal 
or nonverbal, and whether they were used as pre- or 
posttestsoras internaltestsin this study. The пате 
of eachtest and the indices of thinking proficiency on 
which it was scored are also listed. 


ESTS OF PRODUCTIVE THINKING USED IN THIS STUDY 


Name of test Use 
1. Со 


Performance indicators 


mplex Extended Problems with an Emphasis 
ent Thinking (all verbal) 


number of ideas; quality of ideas (two indices); 


Pit problem pretest 


Jewel problem (short form) pretest 


Rare Coins problem internal test 


Jewel problem (long for m) posttest 
Black House problem posttest 
posttest 


X-ray problem 


achievement of solution 


number of ideas, quality of ideas (two indices); 
achievement of solution 


achievement of solution 


number of ideas; quality of ideas (two indices); 
achievement of solution 


number of ideas; quality of ideas (two indices); 
achievement of solution; number of odd facts or 
discrepancies in information that are noted 


number of ideas; quality of ideas (two indices); 
achievement of solution; number of relevant 
questions asked or preparatory steps taken 
before work on problem was begun 
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Complex Extended Problems with an Emphasis 


on Divergent Thinking (all verbal) 


Elevator problem pretest 


The Deep Sea Dive internal test 


The Ancient City posttest 


A Visit to Karam posttest 


3. 


Brief Problems with an Emphasis on Convergent Thinking (; 


number of ideas 


number of odd or puzzling facts within E. utis 
problem that are noted; achievement of s 


indices); 
number of ideas; quality of ideas (two шне | 
number of relevant questions asked or prepa 


tory steps taken before work on the problem 
was begun 


number of puzzling facts which are noted; 
intellectual persistence 


Ingenuity Items (Form A) pretest 


Ingenuity Items (Form B) posttest 


Squares pretest 
nonverbal 
Truck Improvement pretest 
verbal 
Suppose People Could pretest 
Read Minds verbal 
A Summer Carnival pretest 
(Form A) verbal 
Circles posttest 
nonverbal 
Dog Improvement posttest 
verbal 
Suppose There was an posttest 
Intelligence Pill verbal 
A Summer Carnival posttest 
(Form B) verbal 


5. 


all verbal) 


ions 
achievement of solution (number of solution: 
achieved: three possible) 


E 
achievement of solution (number of solution 
achieved: ten possible) 


Test of the Master Thinking Ski 


4. Brief Problems with an Emphasis on Divergent Thinking 


— 
number of ideas; quality of ideas (two indices) 
indices) 
number of ideas; quality of ideas (two indice 
ed 
number of ideas; number of inventions suggest 


" " ail- 
quality of ideas (two indices); number of Ж 
able elements used when creating a produc 


indices) 

number of ideas; quality of ideas (two indice 
indices) 

number of ideas; quality of ideas (two indice 


number of ideas; number of inventions Sug 
gested 


ail- 
quality of ideas (two indices); number of 9 
able elements used when creating a produ! 


ll (verbal) 


The Missing Statue posttest 


total number of “best” 
alternatives to 
rank of one; nu 


or “second best" 
which the student gives a h 
mber of occasions on whic 


Performance Indicators and Scoring 


On each of the various 
of performance were obtained 


ive 
the student chooses the “give up” alternat 


mber of i 


A nu 
applied to performance on such problems. 


owing perfor mance indicators are typical 
Which were used: 


hieve 
У Whether or not the student ас r of 
the rincipal Solution to the problem; the numbe ne 
Odd facts or d 


The fol 
of those 


т ch 
iscrepancies in the problem whi ible 
noticed; the 


ssi 
number of ideas he wrote as ро 


Ме" n be 
ndices of thinking proficiency Са! - 


EJ 
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Quality ratings were made by a team of trained 
Scorers whofirstassigneda separate rating to each of 
a student's ideas ina problem, and then recorded both 
the number of ‘‘high quality"’ ideas and thequality rat- 
ing of the “best” idea which the student produced. 


In addition to measuring the student's performance 
on problem-solving tasks, an attempt was made to 
probe his understanding of the nature of problem 
solving by presenting him with three open-ended sen- 
tences which he wasaskedtocomplete. The sentences 
were: ''Making mistakes in working on a problem 
© ^"; “Wild or silly ideas are n 
an, “In order to solve problems you must 
us answers rated by scorers on a 3- point 
hier indicating the degree of insight or understanding 
pied was shown by the answer. Each of the three 
ace nces was scored as a separate variable and is 

erred to as Sentence I, II, or III, respectively, in 
the analysis. 


; 
"n 


Experimental Design and Statistical Analysis 


ms basic analyses for this experiment were done 
M а 5207 factorial design. The factors repre- 
E were: Instruction (Treatment vs. Control); 
Sa i сөнө (Facilitative vs. Nonfacilitative); and 
Aa is пеле меге eleven classrooms within each of 
XR cells of this design, and standard analysis of 
riance procedures were used for the analyses. 


ERR the investigators also wished to look at 
E re ationships between IQ and the Treatment and 
nvironment factors, a secondary analysis was per- 
formed, using a 3x2x2 factorial design. Three 
levels of IQ were represented in this design--high 
(above 115), middle (101 through 115), and low (100 
and below). The other two factors were Treatment- 
Control and Facilitative-Nonfacilitative Environment. 
After all subjects had been blocked into three groups 
on the basis of IQ, a random sample of twenty was 
Selected at each IQ level within each Treatment x 
Environment combination in the design. Thus, a 
total of 240 subjects was used for this secondary 
analysis. (In order to obtain such a sample, it was 
necessary to ignore classifications according to sex 
and classroom units involved in the experiment. ) 


дак) both the main analysis (the 2x 2x 2 factorial 
ate the secondary analysis (the 3x 2x2 fac- 
varianc esign involving IQ), separate analyses of 
ternal ees performed on each of the pretest, in- 
1. Smilax and posttest measures indicated in Table 
stratification ee. were performed on each of five 
all mean s on variables--socioeconomic status, оуег- 
IQ, Ме еы onthe Stanford Achievement Battery, 
deb rne mu the Soet Studies part of the 
When give equential Test of Educational Progress) 
this test RSS à pretest (Form 4A), and scores on 
en given as a posttest (using Form 4B). 


lie cok main design, the classroom mean score 

the seco ‘joes was the unit of analysis, while in 

Scor nddesign, the analysis was performed using 
es of individual students as the basic unit. 


RESULTS? 


vie Атау ses of variance of the stratification variables 
fere ed no significant treatment-control group dif- 
Nees, Sex differences were found for Stanford 
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Achievement Test (Е=7. 81; df =1, 80; p<. 01) апа 
the Kuhlmann- Anderson IQ scores (F-8. 81; df - 1,80; 
p<.01). Inboth cases, females scored higher than 
males (0. 4 years on the Stanford and 4. 55 points on 
the IQ test). Significant Environment effects were 
found on all stratification variables (Table 2); the 
means are presented in Table 3. 


TABLE 2 
MEAN SQUARES AND F RATIOS FOR SIGNIFICANT 


ENVIRONMENT EFFECTS: STRATIFICATION 
VARIABLES (1 AND 80 DEGREES OF FREEDOM) 


Variable Mean Square F 

Socioeconomic status 24, 04 11,90** 

Stanford achievement 2.09 5. 64* 

Kuhlmann- Anderson IQ 484. 02 9, 3T** 

STEP pretest 41.39 4. 55* 

STEP posttest 42.62 4, 30* 
*р< .05 

**p<.01 

TABLE 3 


MEAN LEVEL OF PERFORMANCE OF STUDENTS 


IN FACILITATIVE AND NONFACILITATIVE 
STRATIFICATION VARIABLES 


ENVIRONMENTS: 


Environment 
Variable Facilitative Nonfacilitative 
Socioeconomic status 2.45 3,50 
Stantord achievement 5.25 4, 93 
Kuhlmann- Anderson IQ 109. 74 105. 05 
STEP pretest 27.18 25.71 
26. 18 24.19 


STEP posttest 


Pretest Results 


'enty-one analyses of pretest scores, only 

н. ee лесі was significant, forthe measure 
of Achievement of Solution on the Jewel Problem 
(Е =5. 46; df= 1, 80; p<- 05). This difference re- 
iority of students in the treatment 


flected the superiority Ot d 
group ( 8 percent achieving the correct solution as 
opposed to only 4 percent in the control group). It 


seems likely thatthis significant F is simply an arti- 
fact of the number of analyses carried out. Table 

4 presents the mean squares and F ratios for the 
significant Environment effects on the pretest meas- 
ures, with the corresponding means in Table 5. 
Tables 6 and 7 present the mean squares, F ratios, 
and means for those variables on which significant 
sex differences were found. The means in Table 5 

indicate the consistently superior performance of 

students in the facilitative environments, and those 

in Table 7 show that males scored higher on three 

measures (all related to the same problem), while 

females scored higher on the other five. 


"2 
TABLE 4 
MEAN SQUARES AND F RATIOS FOR SIGNIFICANT 


: T 
ENVIRONMENT EFFECTS: PRETES 
(1 AND 80 DEGREES OF FREEDOM) 


Variables MeanSquare F 


Squares problem 


Number of ideas 


23. 66 4. нй 
ст а 
Truck improvement problem 
Number high quality ideas 2.04 5.37* 
Quality rating,best idea 0.27 5. 56* 
С MENSEM 
„ә уй! 
TABLE 5 
MEANS FOR STUDENTS IN FACILITATIVE AND 


NONFACILITATIVE ENVIRONMEN 


TS FOR SIGNIFI- 
CANT PRETEST VARIABLES 


Environment 
Variable Facilitative Nonfacilitative 
Squares problem 


Number of ideas 


8.97 7.93 
Number highquality ideas 2.18 1.67 
Quality rating, best idea 1.71 


Truck improvement problem 


Number high quality ideas 1.58 1,27 
Quality rating, best idea 1, 69 


Internal and Posttest Results 
28 78Sttest Results 
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Analyses of variance of the Trees unten) fest. 
measures and thirty-seven posttest variables ae Te 
in one significant F ratio for the Environment e 05) , 
9n the Ingenuity Test (Е -4.89; dí - 1, 80; р. ы 
This isa considerable contrast to the pretest p s 
in which five significant differences (out of eae 
one analyses) were found, allof which indicated AE 
Superiority of students in facilitative environme = 
The notable lack of posttest effects may partly be i 
counted for by the presence of several pue 
Treatment x Environment interactions (see below d 
all of which indicate that treatment was more effec 
tive with students in nonfacilitative environments, 
the initial differences ый зе Ща 
cilitative groups. Several additiona 
Treatment x Environment interactions, though they 
failedto reach Significance, also showed this pattern. 


TABLE 6 


MEAN SQUARES AND F RATIOS FOR SIGNIFICANT 
SEX EFFECTS: PRETEST 


(1 AND 80 DEGREES OF FREEDOM) 


Variable 


Mean Square F 
Mean square F 


Pit Problem 


++ 
Achievement of Solution 0.41 у RH 
Number of ideas 2,21 А 31** 
Number highquality ideas 1.18 9. 07** 
Quality rating, best idea 2.16 13. 
Mind-reading problem 
** 
Number of ideag 15.54 8,88 
Summer carnival 
7* 
Number high quality ideas 0. 47 5.6 


0* 
Quality rating, best idea 0. 47 5.1 


TP 19. 11** 
Ability to Synthesize 5.77 ae 
*p«.05 


ЖЖ astu 
TABLE 7 
I- 
MEANS FOR MALES AND FEMALES FOR SIGNIF 
CANT SEX EFFECTS: PRETEST 
Variable Males — 
Pit Problem 
Achievement of Solution 0. 49 га 79 
Number of ideas 3.47 * 96 
Number high quality ideas 1.19 r 86 
uality rating, best idea 3.17 à 
Mind-reading Problem 
Number of ideas 3.52 adi 
Summer Carnival 
í 48 
Number highquality ideas 0. 33 17 
Quality rating, best idea 1. 03 2.36 
Ability to Synthesize l. 84 i 
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TABLE 8 


MEAN SQUARES AND F RATIOS FOR SIGNIFICANT 
TREATMENT EFFECTS: INTERNAL AND POST- 
TESTS (1 а FOR EFFECTS ) 


Mean 
Variable Square Е 
Deep sea dive problem (internal)? 
Sensitivity toincongruous 
information 2.25 141% 
Jewel problem: 
Number of ideasb 13.46 6.59* 
House problema 
Achievement of solution 0.23 6. 47* 
Number highquality ideas 2.05 16. 99** 
Quality rating, best idea 1.84  5.98* 
Sensitivity to incogruous 
information 1.50 15, 65** 
X-ray problema 
Achievement of solution 0.16  6.20* 
Quality rating, best idea 0.99 5, 22* 
Ancient city problem: 
Number of ideas b 22.66 10, 97** 
Visit to Karam: 
Persistence 2.74 4. 15* 
Intelligence pill: 
Inventiveness a 0.38 4. 33* 
Statue problem: 
Effectiveness of Master 
Thinking Skill 1.97  5.06* 
Understanding thinking: 
Sentence Ib 0.68 6.79* 


*p<.05 
р<.01 


а 
80 df for error 
36 dt for error 


* 


Ea, i 
Ch of the Significant sex differences reported 


in T, Н 
miim 19 twelve in forty analyses) reflects the 
Hbi cs ormance of females. Seven of the twelve 
Production of gased On Variables dealing with the 
Ereater sen ен, three differences reflected the 
Zling intor Sitivity of females to discrepant or puz- 
maining t mation embedded in the tests, and the re- 
measures, W differences occurred on a variety of 
ures, the dat ithrespect to the verbal fluency meas- 
ings that ata are consistent with the numerous find- 
girls are verbally more proficient than boys. 


In the analyses of all these measures- - pretest, 
internaltest, and posttest- -there wasa total of seven 
significant interactions, all in the internal and post- 
tests. Two ofthese, three-way interactions of Treat- 
ment x Environment x Sex, were uninterpretable. 
The other five were Treatment x Environment inter- 
actions for which the means are presented in Table 
11. The interactions for these five measures all have 
the same pattern: withinthe treatment groups those 
students in nonfacilitative environments performed 
better than students in facilitative environments , 
while for the control group, students in the facilitative 
environments performed better than those in the non- 
facilitative environments. Thus, it would seem that 
one important effect of the training materials was to 
enable students from less favorable classroom cli- 
mates to overcome the detrimental effects that such 
environments would otherwise have produced on their 
performance. This may help explain thefact (noted 
earlier) that classroom environment was a significant 
source of variance on five out of twenty-one pretest 
measures, but on only one out of forty posttest 
measures, 


TABLE 9 


MEANS FOR TREATMENT AND CONTROL GROUPS 
FOR SIGNIFICANT POSTTEST EFFECTS 


а ОВ 
Variable Treatment Control 
Deep sea dive problem 
Sensitivity to incongruous 
information 1. 63 1.29 
Jewel problem: 
Number of ideas 6. 67 5.58 
House problem 
Achievement of solution 0. 70 0.59 
Number highquality ideas 1.32 1 01 
Quality rating, best idea 4. 23 3; 94 
Sensitivity to incongruous : 
information 0. 66 0. 40 
X-ray problem 
Achievement of solution 0.11 0.19 
Quality rating, bestidea 2. 48 2. 69 
Ancient city: 
Number of ideas 5.22 4.06 
Visit to Karam: 
Persistence 2.70 2.22 
Intelligence pill: 
Inventiveness 0. 08 0.03 
Statue problem: 
Effectiveness of Master 
Thinking Skill 3.73 3.32 
Understanding thinking: 
Sentence I 0. 78 0 
: . 53 
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TABLE 10 


MEAN SQUARES AND F RATIOS FOR SIGNIFICANT 
SEX EFFECTS: INTERNAL AND POSTTESTS 
(1 df FOR EFFECTS) 


Mean 
Variable Square F 
Deep sea dive problem (internal)? 
Sensitivity to incongruous 
infor mation 1.25  4,11* 
House problem? 
Number highquality ideas 0.57 4.71* 
Sensitivity to incongruous 
information 0.67  7.00** 
X-ray problem: 
Sensitivity to incongruous 
information? 1.70 9, 59** 


Circles: 
Number of ійеаѕа 55.63 9, 37** 
Dog improvementa 
Number of ideas 


57.05 14, 13** 
Number high quality ideas 


1.52  4,55* 

Intelligence pill: 

Number of ideas 6.61 9, 99** 
Summer carnival 

Number high quality ideas 0.98 16.97** 

Quality rating, best idea 1.70 20.49** 

Ability to Synthesize 4.99 15,41** 
Understanding thinking: 

Sentence Ib 0.81 8, 19»* 

*p< .05 
** p< .01 


80 df for error 
b36 df for error 


Effects of IQ 


analysis, 


With the exception of Socioeconomic Status 
formance on each of the Stratification variables was 
Significantly related to IQ (р <, 01. The mean dif- 
ferencesfor these variables are all in the predicted 
direction--high IQ students perf 
Stantord and STEP tests than do mi 
Who in turn perform better than 


IQ group outperformed the others, and the чиа 
IQ group, уі опе smallreversal, was consistent y 
superior tothe lowIQgroup. Significant differences 
were found on twenty-seven of the thirty-eight post 
test measures, nineteen of these beyond the . 01 level. 
(IQ effects on the three internal test measures ШО 
not analyzed.) In all cases, the high and middle x 
groups performed substantially better than the ny 
IQ group. However, performance of the high sus 
middle IQ groups on these posttest measures we 
much more alike (relative to the low IQ group) man 
it had been in the pretest, The absence of any sy" 
tematic IQ x Treatment interaction (see below) A 
gests that this change is not of sufficient magnitude 
to be considered an important result of exposure ы 
theprogram. Asinthe pretest, there was a genera 
rank-order relation between IQ and performance on 
most of the posttest measures. 


x d 

A total of nine significant interactions шн 

in this second analysis, four involving pretest m No 
ures and the other five involving the posttest. 


e 
consistent pattern appears to be present in thes 
interactions, 


DISCUSSION 


Effects of Treatment 
setts 01 Treatment 


Clearly, use of the program led to an inerease 
in the level of productive thinking in this large es 
representative group of fifth-grade students. M 
performance of the group which received training ae 
productive thinking skills surpassed that of the 90, 
trol groupon thirty of the forty internal and post 
measures, Of the thirteen significant aueren TA 
elevenfavored the treatmentgroup. Moreover, as 
Superiority was found at all levels of IQ, and W3 
evident on a broad variety of the measures. 


However, there was a lack of treatment авс 
9n nearly all performance measures for the think” 
Problems, most of which emphasized divergent t 
ing. These brief tests, such as those develope um- 
Torrance, are structured soasto elicit a И 
ber of rather Short, discrete responses on relati for 
uncomplicated tasks, One possible explanation 
the lack of Such effects focuses on the question 0 ills- 
relevance of the training materials to such ski т^ 
Ititisassumedthattne brief tests require less d 
Plex cognitive processes for adequate perfor сү. 
than those Processes taught for in the complex р, їп 
lem Settings of Series One, then what is Jeanne um 
Series One is only marginally relevant to per aent 
ance on these brief problems. Thus, the beaga 
Er oup might not show any marked superiority OF егей 
tasks. Another possibility which must be cons! or 
18 that most children of this age, whether train? pni- 
not, can demonstrate these relatively simple ae o 
tive skills 9n demand (under proper condition" 
Short-term motivation), even though they may гате 
make use of them intheir regular school wor ON 
this is true, training provided by the program inking 
Produce Superior performance on complex m ki 
tasks, where the use of such simple cognitive estu 
and a positive Set) is not sufficient for Sue anne 
Performance, but по obvious benefit from tr? 
would be apparent on the brief tests. 
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Effects of Environment 


The five significant Environment effects on the 
pretest measures all resulted from the fact that 
classes with facilitative environments performed 
better than nonfacilitative classes on the two brief 
divergent thinking problems developed by Torrance. 
On the posttest and internal test measures, there 
was only one significant Environment effect out of 
forty comparisons. This virtual absence of post- 
training Environment effects seems to occur in large 
part because thetreatment was particularly effective 
with students in nonfacilitative environments. Nearly 
à dozen Treatment x Environment interactions (five 
of Which are significant and are presented indetail 
in Table 11) support this conclusion. It should be 
noted that the specialbenefits of training for students 
ìn nonfacilitative classes amounted to more than 
Simply a change of ‘‘set,’’ since some of the signifi- 
cant Treatment x Environment interactions involved 
the kind of complex problems in which no main effect 
of Environment was found, Thus it appears that an 
important effect of the training materials was to im- 
Prove both the cognitive skills and the general atti- 
tudes of students from the less favorable classroom 
environments. 


Effects of Sex 


Significant sex differences occurred on twenty - 
One of the total of sixty-one pretest, posttest, and 
internal test measures used in this study. With the 
exception of three measures in the pretest Pit Prob- 
lem, all significant sex effects resulted from the 


TABLE 11 


superior performance of females. This female 
superiority occurred on all types of problems (both 
verbal and nonverbal, convergent anddivergent), and 
on all performance indicators except measures of 
intellectual persistence and of the achievement of 
solution to problems. This finding of generally su- 
perior performance by females is consistent with 
other findings (e.g., 10) concerning the verbal per- 
formance of students of this age. 


Effects of IQ 


The general level of intellectual ability hada clear 
and significant effect upon performance for most of 
the measures used in this study. In most cases, the 
predictable rank-order relation between IQ and per- 
formance was found, withthe High IQ group obtaining 
Scores superior to those obtained by the Middle IQ 
group, which were in turn superior to those obtained 
by the Low group. The effect of IQ was significant 
on all types of performance measures, and on all 
ty pes of problems except the specialtestof the Master 
Thinking Skill. 


Large and significant IQ effects were found on all 
of the brief divergent thinking problems, andthis 
effect was especially marked on the four Torrance 
tests. These effects were often large in an absolute 
sense as well as significant statistically. This set 
of findings suggests that early reports of a ''virtual 
lack of relationship between measures of creative 
thinking and IQ” (14) are perhaps something of an 
overstatement and that such findings may have resulted 
from sampling restrictions. Since all IQ analyses 


MEANS FOR SIGNIFICANT TREATMENT x ENVIRONMENT INTERACTIONS ( 1 df FOR INTERACTION ) 


df for Environment 
Variable error Facilitative Nonfacilitative 
Rare coins problem 80 
омана] test): 
Chievement of solution Treatment .23 .22 
Control .28 12 
Je 
Nus broblem (posttest): 36 
er of ideas Treatment 6.25 7.21 
Control 6. 03 5.21 
D 
229918 (posttest): 80 
er of ideas Treatment 8.10 8. 46 
Control 8.28 6.86 
N i TE 
umber highquality ideas Treatment 1.20 1.31 
Control 1.34 .92 
Ийан problem (posttest): 80 
iensitivity to 
n Я 
Cogruous information Treatment .94 79 


Control 
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e present study are based upon the carefully se- 
oe of 240 students (described earlier) 
which should be representative of the entire distri- 
bution of ability in a total of forty-four classrooms, 
the highly significant IQ effects obtained cannot easily 
be dismissed аз ап artifact of the particular sample 
used. 


Indeed, it seems reasonable to assume that there 
is a moderate correlation between IQ and creative 
thinking when the entire distribution of intellectual 
ability is considered. Most studies which have re- 
ported the absence of such a correlation (e.g. 5. S; 
15) involved highly selective, nonrepresentative 
samples, while studies which reported a positive re- 
lation between IQ and creative thinking (e.g. , 6, 13) 
have used far more representative samples, 


In contrast to the effect of IQ on test performance, 
there was no consistent pattern of interactions between 
IQ and Treatment; that is, the increments in thinking 
and problem-solving performance produced by the 
training program were approximately equal for all 
three levels of IQ. This finding agrees with the re- 
sults of the earlier Covington and Crutchfield study 


(2), and also with the more recent study by Olton and 
Crutchfield (11), 


SUMMARY 


The major purpose of this study was to investigate 
the extent to which increments in the thinking and 
problem-solving performance of fifth-grade students 
could be produced by the useof self-instructjonal 
programmed lessons designed to teach skills and 
strategies of creative thinking independent of any spe- 
cific subject field. The study also investigated the 
relationshipbetween productive thinking abilities and 
certain characteristics of the learner (IQ and sex) 
andthe relationship between level of productive think- 
ing performance and the extent to which Overall 
classroom “environment”? was judged to facilitate 
creative thinking, 


Results from forty-four fifth- 
that the training materials produ 
nificant increments i 
performance on a 
ing measures. The 
for virtually all types of students ( 
or level of 1Q), i 
dents in classro 


Iced statistically sig- 
Solving 


its occurred 
regardless of sex 


instructional program; consider. 
tional benefits could be expected 
where materials are reinforced 
ticipation, 


FOOTNOTES 


1. The research reported herein was performed 
pursuanttoa contract with the United States _ 
Office of Education, Department of Health, Edu 
cation, and Welfare, under the provisions of the 
Cooperative Research Program (Center No. 
C-03/Contract OE 5-10-154, Wisconsin Re 
search and Development Center for Cognitive 
Learning) with additional support of a grant 
from the Carnegie Corporation of New York 
to the Creative Thinking Project, Institute of 
Personality Assessment and Research, Uni- 
versity of California, Berkeley. 


2. Four tests inthe battery (Circle, Squares, Truck 
Improvement, and Dog Improvement) were de 
veloped by Dr. Е. Paul Torrance while at the 
University of Minnesota. His permission to » 
use those tests as part of this study isgrate 
fully acknowledged, 


3. A detailed presentation of the results of this ex^ 
periment, including tables reporting means, 
standard deviation, and results of analyses of 
all variables will be found in Olton and others 
(12. Further analyses are currently being на 
conducted by Mr. James Ваугу of the Wiscon” 
Sin Research and Development Center for Cog 
nitive Learning, Madison. 
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A Gulde for Preschool Teachers 
In Head Start-Type Programs ot 
Compensatory Education 


EDITED ay 
Robert E. Clasen 


O N TO THE CLASSROOM deals 

with typical problems common to h 
teachers of disadvantaged preschool 
children and contains unique suggestions 
for understanding and meeting the needs 
of these youngsters. The chapters are 
based on papers by well-qualified pro- 
fessors 35 professionals from the 
preschool education field which were 
originally presented to a group of Head 
Start teachers needing help in the various 
areas covered. The editor says, “Since 
these works were extremely useful to one 
Broup of teachers, they should be 
useful to others," 


The 


The book begins with a chapter which 


Creating a Lea 


à teacher's aware 


From a Teacher's Point of View 
to day 
ming in Head Start, plus the h 
classroom from the first class d 
teacher's log with her comment 


^ Conversation with 


200 pages $7.25 Hardcover and $5.75 Softcover 


ming Environme 


nt (numerous hints are given on 
learning climate can 


how this 1, n be created) (Chapter 2) 
Teacher, The Child and Head Start (the needs of 


children and 
ness are discussed ) (Chapter 3) 


Speech Language Acquisition and Language and Head Start (deal 
with language diagnosis and te; 


aching strategies ) (Chapters 4, 5) 


(a humorous and heart-ren¢ 
canvassing, and parent pro; 
happenings in а Head Start 
day to the last—all taken from a 
агу and suggestions ) (Chapter 6) 


ling day 


punt of organizing, gram- 


a Head Start (A.D.C. 


) Mother (reveals what 
jl the mother of a Head Start child experiences ) (Chapter 7) 
fefines “culturally deprived” and offers a 
iden’ Of reference for the thoughts and Programming for Parents (offers surprising views on what this 
lens presented in the remainder of is all about) (Chapter 8) 
the ok, Each chapter was selected 
у Dr. Clasen on one criterion: 1 = ariz 
Does it contain information nich our Qu ends by LASSROQN panes the real purpose of 
Gerionce has shown that teachers need? : shared th n 

е chapte: 


that 
in а 


TS speak for themselves 


PLEASE SEND ME THE INDICATED NUMBER OF 
ШЕ 


hope of cach of us is 
an idea shared through this medium may 


Y stimulate a change 
teacher's. behavior for the benefit of a child." 
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FREE RECALL OF VERBAL MATERIAL IN THE 
CLASSROOM AS A FUNCTION OF METHOD OF 
PRESENTATION AND METHOD OF PRACTICE' 


JAMES R. WHITMAN 


Veterans Administration Hospital 
Tacoma, Washington 


ABSTRACT 


A CLASSIFICATION of the 
Scriptive of theclassroom 
situation would include tho: 
tobe learned, how the task is presented, and the con- 
ditions under which the student Participates in the 
presentation, Subject matter varies, of course, in 
difficulty; but an instructor is frequently making de- 
cisions regarding how the content of a lesson is to 
be presented, i. e., how the presentation is organized 
for the student. He may also have some Control over 


the method used by the student in completing assign- 
ments. 


types of variables de- 
presentation as alearning 
Se which describe what is 


In this study these fa 
terms of variables influen 
of nonsense words consi 


tors were interpreted in 
icing free recall, with lists 
tituting the material to be 
learned. Nonsense words instead of textbook material 
were used since most students are unfamiliar with 
these; and what is more important, recall for this 
type of material involves the learning of the pattern 
of interrelationships (forms of internal Structure, 
Garner 1962) existing between letter variables, Dif- 
ferent methods for the presentation of stimuli also 
have been shown to influence the rate at which cer- 
tain forms of internal structure can be learned (3). 
The task used accordingly is one involving the per- 
ception of relationships in the material learned, and 
the purpose of this study is to investigate the effect 
of performance of varying both the method for the 
presentation of such materialand the method used by 
the learner. Ss received practice on a list of non- 
Sense words, were tested for recall of these, and 
then received additional practice on a second list. The 


CO ЕЕН 


hypotheses tested were tothe effect that (a) studying 
a list would result in a greater number of Wore, 
recalled than would copying the words in thie БАН 
(0) grouping the words according to similarities. 
word structure would result in a greater number n 
words recalled than would an arrangement in Жу" 
Such groupings аге avoided; and (c) the effects Ps 
duced by studying and grouping the words would A E 
fer to a second task for which studying and LU 
Procedures were not used. To add generality to e 
findings, the nature of the interrelationships betwe g 
the letter variables, which constituted the ae ttis 
task, was varied by using two levels of complexi 


METHOD 


General Procedure 
MI Procedure 


The experime 


| nd 
nt was conductedon a single day 2 
luring the ге 


gularly scheduled class times. ists 
method of complete presentation wasused. The li В 
ОЇ words to be learned and all instructions заве to 
by Ss Were contained in a booklet form distribute ion 
S. The first Page of the booklet contained infor ina 
on how the experiment was to be conducted. This eae 
formation was read and explained to the cla d 
Subsequent pages contained the material to be ТЕЗ = 
and all directions Гог. practice (learning period) та” 
lowed by а. lined sheet-with Бае: [op е 
ing answers (answer period). All learning perio c- 
and test periods were timed and each lasted 90 ве! 
Onds. A learning period followed by a test D thé 
Constituted a trial, Two trials were given with t 


mu 
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S's first list, and these were followed immediately 
by one trial with the S's second list. 


EXPERIMENTAL CONDITIONS 


Eight experimentalgroups of twenty-two totwenty- 
three each were used. Each represented one of the 
possible combinations in a 2x 2x2 experimental de- 
Sign. The experimental variables, each with two 
values, were: formof internal structure, i. e., level 
of complexity, order in which the words were listed, 
and instructions for practice, Form of internal struc- 
ture was varied by using one list of four letter words 
in which allof the internal structure was in the form 
of a simple contingency between the letter values 
used in the first two positions of the words (List 1), 
and one list of four letter words in which all of the 
internal structure was ina form whichdid not include 


any simple contingencies between the letter values 
in the words (List 2). 


Grouping was varied in the following manner. 
Words in a list were ordered either with the same 
first (and second insofar as possible) letters together 
(Grouped Order)or suchordering was avoided (Mixed 
Order). Both groupings were used for the first 
learning task. Only the Mixed Order was used for 
the second. The following instructions for practice 
were given for the different trials: For the Study 
Groups on Trial 1: ''Study the list below and try to 
find out the rule or system by which the words for 
the list were selected. The rule should be a state- 
ment which would helpa person learnthe list when he 
Sees it. The order is not important. Do not try 
to remember the words--look for the rule or system. 


Donot write on this page.” For the Study Groups on 
Trial 2: *'Below is the same list. This time study 


the list so that you can remember all of the words. 
Remember the гше, Do not write on this page. '" 


For the Practice Groups on Trials 1 and2: “Сору 
the words inthe list below and try to remember them 
as you printthem, Printthe words in the order given, 
but do not try to learn the order." For all Groups 
On Trial 3: “Below is a second and different list. 
Copy the list and try to remember all of the words-- 
in any order." 


All experimental conditions were used with each 
or 45 Presentation, For each condition separate forms 
En ere were used. The classroom distribution 
(RBIs alee was in a predetermined random order 
ing Paus Ss eight booklets each and with the follow- 
given to S lon. A digit substitution task had been 
Ss receiver ү the beginning of the class session, and 
derin heir booklets on the basis of a rank or- 

Б of their scores on this task. 


THE LISTS? 


‘eae de land 2 were used for the first learning task. 
E Was the following: BROZ, PLAJ, BRAZ 
» BROJ, PLAZ, BRAJ, and PLOZ. List 2 


BROZ, PLAJ, BLAZ, PR 
ani Fed , PROJ, BLOJ, PRAZ, 


was: 
BRAJ, 


or th / i A : 
and List oe groups who received practice on List 1 


Spectively. List la was: BLAZ, PROJ, 


List 
taski 5 laand 2а were used for the second learning 


BLAJ, PROZ, BLOZ, PRAJ, BLOJ, and PRAZ;and 
List 2a was: BRAZ, PLOJ, BLAJ, PROZ, BLOZ, 
PRAJ, BROJ, and PLAZ. The form ofinternal 
structure for List 1 is identical to that for List la; 
and that for List 2 is identical to that for List 2a. 


SUBJECTS 


Ss were 182 students in four undergraduate psy- 
chology courses. They had been told a week before 
thetime of the experiment that an experiment in learn- 
ing would be conducted during one of their class 
periods. 


RESULTS AND DISCUSSION 


S's score was the number of words correctly 
recalled in any order. On Trial 1, Ss who were in- 
structed to study the list were required to state a 
rule. This rule was not scored. Those who copied 
the list were required to reproduce the list. Com- 
parisons between groups were оп ће basis of number 
of S's correctly recalling all eight words in the list 
on Test Trial 2 which was thefirsttest trial in which 
recall was required for all groups. These data are 
shown in Table 1. Also the data, expressed as per- 
centages of Ss, and grouped according to the experi- 
mental variables, are shown in Figure 1. Chi-square, 
corrected for continuity, was used to determine the 
significance of differences; andthe p ~ . 05 level was 
used. = 


The two lists differed in level of difficulty, and 
comparisons were made for each list separately. Also 
comparisons were made between pairs of groups as 
shown in Figure 1. For the less difficult list (List 
1), the number of Ss giving all correct responses was 
greater withthe Grouped than with the Mixed presen- 
tation, but the difference on this measure between 
Study and Copy Method was not significant. For List 
2, the more difficult list, the difference between the 
Grouped and Mixed presentation was also significant 
and in the same direction, With this learning task 
however, more Ss gave all correct responses with 
the Study Method than with the Copy Method. 


Ss received practice ona second list, S's scores 
after practice on this list are shown in Table 2, For 
List la, comparisons were made on the same basis 
as that used for Lists 1 and 2 above; but for List 2a 
the comparisons were on the basis of mean їйрє 
of correct responses per group since only twelve о » 
of ninety Ss reached the criterion of all correct x 
sponses; and an analysis of variance and the E EU 
level were used with these data, Also, К S08 


comparisons 
were made for eachlist separ ately. parisons 


The diffe Ces 
between the groups receiving practice on List M. 
E re 


not significant; also, the differences 

ant; S Y ее 
groups receiving practice on List 2a ыча з n A 
cant. The finding, therefore, is that the Mie ney 
duced by studying the grouping of the Mende paga 
first list were not transferred to a second tai ud 

ki х as | 
which studying and grouping procedures were not d 

sed, 

| These results lead to 
is that for a task re pihe tirst 
ariables the order orarr tiene 
This result extends 


1 and Garner 
or presentation 
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of positive and negative instances can increase the conclusion, therefore, is that the efficiency of the 
difficulty of aconcept learning task. The study method, study method is related to the level of difficulty of 
i.e., looking for a rule rather than copying the ma- the learning task. 

terial whiletryingto memorize, produced more cor- 


rectresponses withthe difficultlistonly. The second The transfer effect reflected in the performance 


TABLE 1 


ECT (ITALIC) 
ENT OF Ss GIVING ALL CORRECT RESPONSES (ROMAN) AND MEAN PERCENT CORR 

хова PRACTICE ON THE FIRST LEARNING TASK. GROUPING IS BY MATERIAL IN LEARNING ТАЅК, 
ARRANGEMENT OF THE MATERIAL, AND BY PRACTICE INSTRUCTIONS2 


Material in Learning Task 


List 1 List 2 
Practice Instructions: Study Copy Study Copy 
Grouped 87 96 61 23 
Arrangement 97 99 72 68. 
of Material 
Mixed 61 61 26 14 
88 92 61 E 


aN - 23 for all groups except for the two groups copying List 2, where N- 22. 


FIGURE 1 


SPONSES FOR COMPARISO 9 ENT- fz 
AL VARIABLES (N-92 FOR LIST 1; N - 90 FOR LIST 2.) HONS ВАЗЕР ОМ THE EXPERIMEN r 
100 
80 
uU 
Ф| 
ш во 
о 
= 
a 
ш 
сә 40 
a 
ш 
а. 
20 
0 E] 
G M 
6 M 5 с 
ORDER OF PRESENTATION AND PRACTICE INSTRUCTIONS 
G= Grouped M=Mixed S=Stud C=C | 
еа y = Copy 


" 
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TABLE 2 
PERFORMANCE OF GROUPS ON TRANSFER TASK PERCENT OF Ss GIVING ALL CORRECT RESPONSES 


(ROMAN) AND MEAN PERCENT CORRECT (ITALIC) AFTER LEARNING TRIAL. GROUPING IS BY EXPERI- 
MENTAL CONDITIONS UNDER WHICH FIRST LIST WAS LEARNED a 


Material in Learning Task 


List 1 List 2 
Practice Instructions: Study Copy Study Copy 
Grouped 70 70 17 5 
Arrangement 84 19 57 56. 
of Material 
Mixed 70 61 13 18 
83 82 59 56 


ам = 23 for all groups except for the two copying List 2 as the first task, where N - 22. 


ОГ thedifferentgroupson the second learning task is REFERENCES 
more difficult to interpret. It is of considerable im- 
portance to deter mine the extent to which the results 


Obtained are a function of the amount of practice, i. e., 1. Garner, W.R., Uncertainty and Structure as 
number of lists learned. Psychological Concepts, Wiley, New York, 1962. 
FOOTNOTES 
2. Garner, W.R. , and Whitman, J.R., “Formand 
1. A portion of this paper was presented at the 75th Amount of Internal Structure as Factors in Free- 
Annual Convention, APA, 1967. recall Learning of Nonsense Words, ” Journal 
of Verbal Learning and Verbal Behavior, 4:257- 
2, For a detailed description of the Lists used in 266, 1965. mum j 


this study see Garner and Whitman, 1965, 


3. A significantly greater number of Ss copying the 


ordered List 1, as contrasted to the mixed 3. Whitman, J.R., and Garner, W.R., ‘‘ Concept 
List 1, pars all correct responses on this Learning asa Function of Form of Internal Struc- 
trial (X2- 7.393, df=1). No S copying List ture,"  Journalof Verbal Learning and Verbal 
2 gave all correct responses. Behavior, 2:195-202, 1963. 
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IS THE PHONETIC GENERALIZATION 
HYPOTHESIS IN SPELLING VALID?’ 


ALBERT H. YEE 


University of Wisconsin, Madison 


ABSTRACT 


This study was conducted to help settle a major controyers 


generalization hypothesis, 
girls versus boys, pretesting versus попрге! 
pendent variables were two Spelling test score 


better than nonpretesting, and questionable a 
Study methods appear to be m 
tain the definite value of Test-Study methods 


ing generalizations 
regarded only as one Supplemental aid, 


some authorities fee] that English- 


argued: ‘Sequential training in helping children to 
listen for phonemes in relation to their placement in 
words, to the way in which they are enunciated, and 


Although Spelling has becom 
less concern than others, 
matics, and social Studies 


е a content area of 
such as reading, mathe- 
, it continues to be an 


S showed girls superior to boys, pretesting 


Anis с елы i st- 
phonetic instruction over no phonetic instruction. Te 


k + scer- 
Further research is necessary to asce 


A factor analytic study 
by Allen and Ager (1) found that Spelling is an inde- 
pendent skill and **should receive specific attention 
curriculum" (1:159). 
The general public perceives the ability to spell a $ 
à mark of educational achievement and often evalu- 


ates the success of Schools by the spelling proficiency 


Spelling is important because it in- 
chase of millions of expendable work- 
books each School year and has become big business. 
As expected, such purchased materials reflect vary 
ing positions between the two points of view. А180, 
despite its lower status as а content area, a signifi 


in spelling, ils. 
fects all elementary school teachers and their pup! 


We conducted this study to ascertain the value ої 
Providing phonetic generalizations in spelling ins o 
Thus, we hypothesized that spelling SCORE 
Pupils taught phonetic generalizations as eee 
a leading Spelling series willbe greater than sco 
of pupils not taught phonetic generalizations. 


Е nt courses of action (i. €., sed 
in inferential trees with branching alternatives) ne 

to be tested by crucial experiments. In the area © 
Spelling instruction, it is important to know if pho t 
netic Seneralizations do or do not make a significa” 
difference in Spelling achievement, because nee 
ling Controversy has in fact reached the branching 
point. Empirica] research needs to ascertain VE is 

Seneralization hypothesis, Thus, i 


Or nay to the 
important to see that differing implications follow 


either Outcome, 


EM 


чыё 


METHOD 


Subjects 


In the Fall Semester of 1965, all 106 elementary 
Student teachers at the University of Texas and their 
pupils in grades 2 to 6 in Austin, Texas schools were 
recruited for this study. Grade one classes were 
omitted from the study because they lacked extensive 
formal spelling instruction. Since student teacher 
assignments in elementary grades 2-6 were in all 
types of schools in the district, the distribution of 
elementary grade classes included a fairly random 
representation of pupils from the school district. The 
twelve schools in which student teachers were loca- 
ted had at the time of testing a total average daily 
attendance of 6,063 pupils of which 2,839 (posttest N) 
Pupils participated in the study. 


a was possible to evaluate the sample drawn for 
this study with objective information from University 
ànd school records in relation to student teachers' 
and pupils’ characteristics. Froma survey of such 
information, the sample was judged representative 
of typical teacher candidates and pupils in urban 
Centers and offered no systematic bias. This sam- 
ple, therefore, makes it possible to generalize re- 
sults and conclusions more readily to similar situa- 
tions elsewhere. 


Factorial Design 


The student teacher-class pairs were randomly 
srouped into the experimental framework of Solomon's 
Four-Group design (21), which Campbell and Stanley 
(3:194-195) have recommended as а “true experi- 
mental design. ” 


Subjects were randomly assigned to one of four 
groups. Group A represented the experimental group 
where pupils were given pre- and posttests and in- 
Struction in two selected phonetic generalizations in 
the interval between testing occasions. Group B rep- 
resented the first control group where pupils were 
given pre- and posttesting but no phonetic instruction, 
Groups C and D received only posttesting, but treat- 
ment group C was taught phonetic rules and their ap- 
plication prior to testing and control group D was not. 


Therefore, the study's experimental design is as 
follows: 


Pretest Posttest 
Experimental Group A 01 x 05 
Control Group B 03 04 
» Control GroupC x 05 
Control GroupD 06 


Where R = randomization, 0 = test observations, and 


x = treatment, i.e., instruction in phonetic generali- 
zations, 


The posttest scores of ils i 
re pupils in the four groups 
uoc. онеге! to class means, and analysis of vari- 
"mee s p to test the statistical null hypothesis: 
4 = 95 = 06. With two levels each for the 


83 


independent variables of pupil's sex (X), pretesting 
(Y), and phonetic treatment (Z), a 2x 2x2 factorial 
analysis was conducted for each grade level. Simul- 
taneous analysis of all grade levels together was not 
done, because the tests were tailored for eachgrade 
level and cannot be considered parallel. 


By designing our experiment with independent vari- 
ables other than Z, we reasoned that the competing 
potencies of sex differences (X) in verbal ability and 
of pretesting (Y) need to be considered. Sex differ- 
ences have been a familiar source of significant vari- 
ance in verbalability. For the latter, there has 
been strong evidence to support the Test-Study and 
Corrected-Test approaches (e.g., 5, 12), which 
place great weight on pretests to help pupils find 
words that require their concern and study. Such 
methods, however, have been slow togain prominence 
in spelling and class instruction. Perhaps the lag 
comes from the common conformity to spelling work- 
books, 


Dependent Variables and Treatment 


A spelling test was developed for each grade level 
with three categories of words: (1) Phonetic spel- 
lings that followed two phonetic generalizations 
taught the particular grade level; (2) Tricky spel- 
lings that nearly followed the generalizations taught, 
but they would be incorrectly spelled if the rules 
were followed literally; and (3) No relationship spel- 
lings that lacked relationship to the generalizations 
taught treatment groups. Thus, each category of test 
words formed a separate dependent variable. Second 
thoughts made us believe that a clear distinction be- 
tween tricky and no relationship words was not fully 
satisfactory for all grade levels. The F results were 
essentially identical for the categories, and no real 
gain was achieved by separate analyses. Reliability 
would be increased by combining the two word lists, 
thus they were classed together to form one w ord 
category which we termed unrelated words, i.e., 
words that cannot be successfully spelled by applying 
the generalizations, 


Table 1 presents the test words for each grade 
level by showing the spellings consistent with the 
phonetic generalizations taught in lower-case letters 
and the spellings unrelated to the generalizations in 
upper-case letters, 


A noted authority in spelling, T. D. Horn, helped 
prepare and check the test words and placement in 
the categories. Table 1 does not list the words as 
they were presented in the test. The words were 
randomly distributed in tests and are grouped ac- 
cording to their categories in Table 1 for the con- 
venience of the reader, 


Lessons for two re 


presentative generalizatio 
were chosen for each e 


grade level from г: 
Ware's (13) Basie Spellin Goals a e 
mize treatment effects, daily lessons Were takenex- 
actly from the series with the Standard week's work 
given to each generalization. Words used in the les- 
Sons were not used in the above Spelling tests, The 
lessons of the popular Kottmeyer and Ware series 
exemplify the phonetic appro: 
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TABLE 1 


GRADE LEVEL TEST WORDS 


2 3 4 5 6 

three babies lately music er 

eat factories totally pilot garag 

feet coming electricity glider agaon as 

ride cherries gravity title revoluti 

rode having colony syllable rotation 

make making freely article projection 

tail daisies queenly radar dangerous 

seat countries lively locate engage 

ate riding exactly cable fudge 

goat caring century startle edge 

tea families discovery giggle promotion 
deep ponies equally lemon fiction 

game funnies uselessly lazy grudge 

bike bodies evenly above damage 

nine closing geography against invention 
gave leaving army pebble sponge 

seed moving fifty bugle dodge 

need skating poetry Scribble attraction 
kite Skies purely ahead conjunction 
stove jellies library grumble courage 
cake ladies fairly rectangle objection 
Soap using unkindly triangle selection 
train writing usefully sugar recreation 
Cone hiding loosely potato college 

take candies loudly stable ADJECTIVE 
neat bunnies hobby triple INJURE 
near Shaking proudly reptile OBJECT 
pail taming energy banana FORTUNE 
ТҮ е» МОУТЕ PRISON SOLDIER 
oke storing PRAIRIE AVENUE OR 
HEAD PLAYS RECIPE EXERCISE ANGE LIC 
THREE PAYS ABLY OVEN LONGITUDE 
VIEW FREEING MACARONI THEATER SERGEANT 
COME LYING LUCKILY TOTAL COMPLEXION 
LOVE TURKEYS 

ROOT UNSTEADILY CRUEL PROFESSION 
SAID DYING TRULY FINAL MISSION 
HAVE ae " DISAGREE SHOVEL EXCLUSION 

Y: WHOLLY SPECIAL OCCASION 

ONE HOEING VALLEY 

PIECE SHOEING 


Grade 2: (1) When two 


Vowels are to, eth 
short words, the first vowel ‘says its name"; n 
on 14 and 15), 
er letter (conso; 


the second is silent (Less 
Isee a vowel, then anoth 
then an e at the end of a short word, 1 


> I hear the name 
of the vowel and the e is silent (Lesson 16). 


Grade 3: (1) For words which end in 
ing (Lesson 20 
the plural by cha; 


drop the e before we add - 
words which end in y form 
Y to i and adding es (Lesson 26). 


Grade 4: (1) The Sound of i at the en 
is usually Spelled by the letter У; when y follows а 


ey, the y is silent (Lesson 


Vowel, as in words like ki 


silent e, we 
(2) Many 
nging the 


d of a word 


27). (2) -ly is added to root words without chang- 


ing the spelling of that root with the exception of 
truly (Lesson 31). 


Grade 5: (1) Many words end with the -e1 sound. 
Words ending with this sound often have the - а 
Spelling (Lesson 26). (2) Words in the vowel-co 
sonant-vowel pattern are usually divided between 

€ first vowel and consonant, the second vowel 
going with the Consonant (Lesson 32). 


Grade 6: (1) The soft g or j sound at the end of 
а word is usually spelled kia E dge as in rage and 
Judge (Lesson 13. (2) The noun suffix -tion is 
usually pronounced Shen, -sion as zhen, and -ion as 
yen (Lesson 2j). Р 


Procedures in Testing and Treatment 


Mechanical aids, such as tape recorders, were 


A 


YEE 


not used to help control treatment and test conditions. 
Providing the more natural classroom situation was 
deemed less of a threat to internal validity thanthe 
possibility of ‘‘novelty effect’’ and limitations in pu- 
pil feedback during treatment sessions. To help en- 
sure uniformity in test administration, each spelling 
test was printed according to the normal procedures 
for spelling tests. In other words, the tests pre- 
sented the word to be pronounced, a sentence in 
which the word was used, and then required that the 
word be read again, 


We took care to avoid contamination of the data 
by the student teachers by informing the groups of 
student teachers only what was necessary for them 
to know at each stage. Three special orientation ses- 
sions were held for the following purposes and groups: 
(1) pretesting of Groups A and B; (2) treatment for 
Groups A and C; and (3) posttesting of all four 
groups. For example, a few days after the pretests 
of Groups A and B were collected, the teachers of 
Groups A and C met together in a group meeting 
where they were given daily lesson plans for 2 weeks’ 
classwork. They were instructed to teach the les- 
sons as printed and demonstrated to them in the 
group meeting. After a period of about 4 weeks af- 
ter pretesting, posttests were administered to all 
groups. The use of class means in analyses also 
helped control the teacher variable. 


Thus, an interval of about 2 weeks separated the 
completion of phonetic instruction in treatment groups 


and posttesting. This time interval was provided to 
allow an objective assessment of the treatment effect 
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more independent of the treatment sessions. The 
treatment effect would be maximal immediately fol- 
lowing phonetic instruction, and we did not want to 
invalid the posttests by the influence of immediate 
recall and mental set. Since the generalization hy- 
pothesis assumes long-term effect, the 2-week inter- 
val seems to be more than fair in terms of forgetting 
and reasonable in terms of overcoming immediate 
treatment effect. 


RESULTS 
Pretest Scores 


To ascertain if there was any bias resulting from 
the randomization process, the difference between 
pretest scores for Groups A and B for each grade 
level was tested with 2x 2 analysis of variance. The 
factors were pupil sex and group (GroupA vs. Group 
B). The only significant results obtained were in 
Grade 6 where girls scored higher than boys with 
phonetic and unrelated words. Since no difference 
between Groups A and B was significant (most of the 
Е ratios were less than one), the experimental group 
and the first control group was equivalent inrespect 
to their responses to the dependent variables prior 
to treatment sessions. 


Reliability 


The internal consistency of the pupils’ scores 
was estimated with Guttman’s (6) L4 formula with 
odd-even split-halves. All coefficients for whole 
class, grade-level total test and phonetic word scores 


MEANS AND STANDARD DEVIATIONS FOR PHONETIC WORDS 


Grade 


Group Class Girls 
Level N =e 
M SD M sD 
à 5 20, 38 1.74 16.78 3.17 
" Б 5 21,97 5. 44 18. 66 5. 63 
С 6 20. 06 1.74 11.61 4. 50 
Б 6 19. 55 2.16 16. 65 1.58 
A 6 19.07 2.39 
" B 5 16. 60 3.42 is 30 B % 
С 4 17.31 1.85 13. 99 3.3 
D 5 11. 90 3. 48 9. 66 3. 98 
A 5 15.36 3.14 
А в 6 14. 90 3.17 th "3 es 
с 4 13.38 3.00 14, 01 i 
5 13. 18 2. 49 10. 95 ^ oe 
A 6 18.01 | 
j 5.31 
" B 5 19.52 6. 41 20. i ФР 
с 1 20. 39 2.01 18.10 ra 
5 18. 98 7.08 16.57 np 
A 5 18.17 : 
; 1.09 
А B 5 19.19 2.18 UEA co 
с 5 18.81 1.32 ES Es 
6 16. 65 3.53 14. а y D 
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TABLE 3 


ANOVA RESULTS FOR PHONETIC WORDS 


df MS 3 
Grade Source 
+ 110.71 Bi Bp 
Pupil sex (X) 1 7.47 agg 
Pretesting (Y) 1 1.30 21D 
Treatment (Z) 1 17 .06 
хү 1 ов aa 
xz 1.6 
2 YZ 1 21.05 . 004 
1 .04 
XYZ 36 12, 49 
Within Groups 1 20. 20 
Between Groups 43 13.75 
Total - 
х 1 115. 75 11.17 ** 
Y 1 64, 81 К Ба 
2 1 98. 38 9, 49 *® 
А xy 1 4.22 41 
XZ 1 5.71 56 
YZ 1 28.24 2.72 
XYZ 1 ‚49 % 
Within Groups 32 10. 37 
Between Groups 7 45.38 
Total 39 16. 65 
x 1 5. 03 : T0 
Y 1 35.35 4.91 
Z 1 6. 38 .89 
4 xy 1 1.12 Ir 
XZ 1 5. 97 -B3 
YZ 1 2. 96 “41 
XYZ 1 9. 38 1.30 
Within Groups 32 7.20 
Between Groups 7 9.47 
Total 39 7.61 
x 1 21.59 1.33 
Y 1 02 + 002 
3i 1 2. 98 -18 
1 10. 61 66 
5 XZ 1 2. 45 15 
YZ 1 44. 54 2.15 
хү? 1 3.07 19 
Within Groups 38 16. 19 
Between Groups 7 12. 18 
Total 45 15.57 
x 1 68. 54 п. 
Y 1 9. 74 1.97 
2 1 01 . 002 
6 ХҮ 1 001 . 0002 
xz 1 2. 97 . 48 
YZ 1 27.39 4.41 
XYZ 1 001 . 002 
Within Groups 34 6.21 
Between Groups 7 15. 52 
Total 41 7. 80 
"P< 025 юре о 


Were in the 90's. 
tic word lists, the 
but adequate TS averagi 


f ndicate the i 
pupils’ measures, 


Phonetic Test Words 
E fest Words 


Table 2 Presents posttest Phonetic word means 
and standard deviations for girls and boys in the 
four groups, Some heterogeneity of variance canbe 
observed in some but not all grade level SDs, but this 


YEE 


TABLE 4 
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MEANS AND STANDARD DEVIATIONS FOR UNRELATED WORDS 


Grade Class Girls Boys 
Level Э Ч 
л (oup xN M sD M SD 
A 5 6.17 31 5. 03 83 
2 B 5 6. 61 1.28 5. 90 1.32 
(е, 6 6. 45 .70 5. 53 .82 
D 6 5. 98 .91 4, 95 35 
A 6 4.92 .18 3.53 61 
3 B 5 5.07 1. 62 3.70 .99 
C 4 4.41 .50 3.47 .68 
D 5 4.51 . 67 2.99 .60 
A 5 3. 68 95 3.27 61 
4 B 6 3.78 1. 13 3. 49 „55 
C 4 3:21 111 3,57 ‚16 
р 5 3. 66 89 2.41 71 
А 6 7. 06 2.40 6. 75 2.20 
5 B 5 1.52 ‚14 1. 65 1.14 
c 7 8.01 .84 7.32 .92 
D 5 1. 95 2.44 6. 43 1. 66 
A 5 9.34 1, 06 6. 99 1.86 
6 B 5 10. 15 Lt 8. 40 .91 
С 5 9.26 „10 7. 45 1.67 
р 6 Weta 1.15 6. 46 1.95 


Problem is offset by establishing a higher level оуег- 
all required for statistical significance, i.e., .025 
Instead of , 05 (14:86). 


The effect of heterogeneity of variance upon re- 
Sults was tested in preliminary analyses by equali- 
zing all Ns by randomly selecting out classes in 
(0:519) larger than the cell with the lowest class N 
those d Since those F results were equivalent to 
тев tained with unequal Ns, we report the latter 
gram E with Veldman's (22) AVAR-23 pro- 
of vari Ch can handle triple-classification analysis 

‘nce (ANOVA). 

A 
бани bon Seen in Table 2, all but two of the dif- 
vor the uiris a means for the pupil sex factor fa- 
Pupil sex de In Table 3, results for the factor of 
for dj ow three Е ratios surpassing the .01 level 


i ferences dott = 
nifi Е great enough to be statistically sig- 
icant in Grades 2, 3, and 6. dins 


F ч 
ation” pretesting factor (Y), one significant F 
S obtained (Grade 3), and all differences be- 


twee 
n means sh ч 
Pretestoq Brus, pretested groups superior to non- 


Ont — 
found 5,016 Significant F ratio (Grade 3) was also 
for y, һе Ке treatutént factor (Z); but unlike results 
Uniform ferences between means indicate no 


Pattern suggesting a direction of effect. 


| Thus, in accounting for variance in pupils' phone- 
tic spellings, the results indicate that the most po- 
tent contributor to variance is pupil's sex, with the 
milder positive effect of pretesting second, and the 
uncertain effect of phonetic instruction least signifi- 
cant. No significant two- or three-way interaction 
effects were found with phonetic words. 


Unrelated Test Words 


Table 4 presents posttest unrelated word means 
and standard deviations for girls and boys in the four 
groups. All but two of the means for girls are 
greater than those for boys. 


Table 5 ANOVA results with unrelated words als 
show that factor X accounts for more of the varian A 
in spelling behavior than Y or Z. These highly si s 
nificant F ratios indicate the superiority of Spelin 
ability by girls over boys. ng 


No significant F ratio was found f 

Y and Z. While the pretested E See oe oe 
than nonpretested groups in all grade levels i М 
was no advantage in contrasts between phonetic tro 
ment and non-treatment groups. Such distincti = 
are evidenced by all F ratios for the Z factor bein 
less than one and three of the five F ratios f "e Y 
factor being greater than one. SRI ASIE RR SI 
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i i Table 3) 
i i i reaches the . 04 level with phonetic words ( a 

TEAC EC УУ не ЕҢ сарн c in grade 6. Although it is considered xia y NM 
Graders PE EFAN anen im uM achieved nonsignificant, the ranking of the means were i D 
Be 017) retlects-hteher RUE a effects pre- cal to those found with unrelated words. Pretes ДЕ 
SE a Er D Жеш RP effect alone produced the highest means, so the pattern 
sent or absent. si 
TABLE 5 


ANOVA RESULTS FOR UNRELATED WORDS 


Grade Source df MS Е 
Pupil sex (X) 1 9. 64 13. g2** 
Pretesting (Y) 1 39 56 
Treatment (Z) 1 07 10 
xY 1 . 002 003 
2 XZ 1 09 12 
YZ 1 3. 64 5.14 
XYZ 1 17 .24 
Within Groups 36 «1, 
Between Groups 7 2.00 
Total 43 .92 
x 1 16. 67 21.67% 
Y 1 2. 09 2.12 
3 2 1 002 . 003 
XY 1 - 06 .07 
XZ 1 18 -24 
YZ 1 31 - 40 
XYZ 1 .23 30 
Within Groups 32 iTi 
Between Groups 7 2.79 
Total 39 1.13 Є 
гаа m 
Y 1 1.09 1. 48 
2 1 “11 „14 
4 ХҮ 1 .03 .04 
xz 1 1.30 1.76 
Үд 1 ‚59 80 
XYZ 1 1. 90 2.51 
Within Groups 32 14 Е 
Between Groups 1 И 93 
Total 39 EZ 
x 1 3. 99 45 
Y А -38 M 
2 1 12 ‚04 
5 xv 1 2.87 1.04 
XZ 1 11 . 04 
YZ 1 3.15 1.36 
XYZ 1 1,11 ‚40 
Within Groups 38 2.16 
Between Groups 7 1.76 
Total 45 2. 60 
x 1 33. 42 13. 92** 
Y 1 10. 41 4.34 
Z 1 . 07 .03 
6 XY 1 71 .30 
XZ 1 . 88 .87 
YZ 1 14. 74 6. 14* 
XYZ 1 . 001 . 0003 
Within Groups 34 2. 40 
Between Groups 7 8.61 
Total 41 3. 46 
*p« .025 жр < 101 


FIGURE 1. 
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J ELATED WORDS. 
SIGNIFICANT YZ INTERACTION IN SIXTH GRADERS' SPELLINGS OF UNR 


9:5 


9.0 


8.5 


7.5 


TREATMENT 


NON- 
TREATMENT 


7.0 
PRETESTED 


Éroup means goes as follows: B» С>А >D. à E. 
ough results for other grades lack significan 

Or YZ interaction, the same hierarchial order dd 

be generally found to support the predominance of suc 

а pattern. Figure 1 illustrates this predominant pat- 

tern with sixth graders’ XY results with unrelated 

Words, 


F 1 illustrates the significant departure of 
Sie ы parallelism, Group B, which meceived 
Pretests but no phonetic treatment, scored highest; 
Group C scored second highest and was the treat- 
ment group which was not pretested; third in line is 
Group A, the experimental group which received both 
Pretesting and treatment; and last comes Group D 
Which received only posttesting; i.e., B» C^ A» D. 
This resulting pattern indicates that either main ef- 
ects, Y or Z, benefited pupils' spelling scores more 
than both of them together; but Y and Z together pro- 
duced better results than when both were absent. 


NOT 
PRETESTED 


DISCUSSION AND CONCLUSIONS 


The hypothesis that 
a positive difference in 
confirmed in this study. 
design this study utilized 
gest that pretesting may 
ling proficiency than pho: 
not investigated the effe. 
would tend to Support t 
This can be seen in Ta 
means for Group C ar 


phonetic instruction provides 
Spelling achievement was not 
The Solomon Four-Group 
makes it possible to sug- 
be more beneficial to Spel- 
netic instruction. If we had 
ct of pretesting, the results 
he generalization hypothesis, 
bles 2 and 4 where most of the 
€ greater than those of GroupD., 


The predominant pattern of 
the following general order: 
either pretesting in Group B 
in Group C appears to be gen 


group means goes in 
B*C^A'D,. Since 
or phonetic instruction 
erally superior to both 
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r (Group A), a choice between pre- 
ie аа А in.favor of the for- 
mer seems feasible and deserves further study. iua 
potential benefits of pretesting appear to go ж 
the range of phonetically-regular words, ee e _ 
results support and complement findings for the Test 
Study methods which emphasize pretesting procedures, 


Let us attempt to explain such findings: Pretest- 
ing may create a mental set toward words that is 
sufficient to produce gains over time. Perhaps the 
mental set produces a dissonance process (4) which 
requires tension-reduction by the pupil s learning 
words that were recognized as difficult. Phonetic 
instruction alone may provide the pupil some prepar- 
atory process to attack unfamiliar words and thus 
help him perform slightly better than those with or 
without Y and Z together, But used as blanket rules, 
they lend themselves to more errorful spelling by 
perhaps creating less dissonance or concern on the 


part of pupils to check and Concentrate upon the Spel- 
ling of individual words. 


Spellings, we think th 
greater probability for 
may bea Confounding of influence 


Z cancel out each Other. Further research is neces- 


Sary to ascertain the validity of such findings. If 
Y and Z effects are found to conflict in later research, 
it would be important to learn why they do, 


The most potent Source o 
Pupils' sex, It should not b 


dized achievement and int 
to Anastasi (2:493): 


In general, girls surpass boys in those 
school Subjects 


bal abilities, memory, 


We need to accept sex differences 
achievement for w 


is beginning to show that teachers often 


in spellin [4 
More research 
do not ade- 


It may not be worth- 
Spelling programs ac- 


methods in Spelling programs after Specific behav- И 
ioral objectives have been stated and carefully evalu 
ated in terms of what comprises efficient spelling 
behavior. Then alternative approaches to handle the 
differing spelling needs of Pupils can be tested. With 
such preliminaries, Systematic instruction can be 
developed to better effect such behavior (16, 17). 


Since the positive effect of ten lessons could not 
be detected in a pattern of superior spelling by treat- 
ment groups or in any significant Fs, the predomi- 
nant use of generalizations for spelling instruction 
appears tobe less useful than Test-Study met ho ds. 
Test-Study methods appear to provide more efficien- 
cy and economy than phonetic instruction, 


FOOTNOTES 

1. The Research Committee of the Graduate School, 
University of Wisconsin, Madison provided 
research support for most of this work. The 
National Science Foundation also hel ped 
through their grant to the University of 
Wisconsin Computing Center. The Research 
and Development Center for Teacher Educa- 
tion, University of Texas, Austin supported 
data processing assistance for preliminary 
analyses. The author thanks Carl Personke 
for a critical reading of an earlier draft, but 
absolves him of responsibility for any short- 
comings of the present version, 


2. Loyal and helpful assistants made this 


5 study pos- 
Sible: Fthel Viest led the group meetings 


and assumed major responsibility for data 
Collection, H, Albert Napier, William 


» and Troy Klein 
and programming 
assistance from Janice 


REFERENCES 


l. Allen, Doris, and Ager, Meu, МД Factor Analytic 
Study of the Ability to Spell," 


i Educational and 
Psychological Measurement, 25:153-161, 1965, 


2; Anastasi, Anne, 
vidual and Group 
ed. ), Macmillan, 


Differentia] Psychology: Indi- 
Differences in Behavior, (3rd 
New York, 1960. 


3. Campbell, D. T., and Stanley, J.C., '*Experi- 


i experimental Design for Re- 
ing," in N. L. Gage (Editor ) 
arch on Teaching, Rand 
0, 1963, pp. 171-246, 


Festinger, La 
ance, Stanf 
alifornia, 


A Theory of Co nitive Disson- 
ord University Press, Stanford, 
1957, 


5. Gates, A. L; “An Experimental Comparison of 
the Study- test and Test- study Methods in Spel- 


ling," Journal of Educational Psycholo; 22: . 
1-19, 1931. c— 


10. 


п. 


12, 


13, 


14, 


YEE 


Guttman, L., “A Basis for Analyzing Test- 
retest Reliability," Psychometrika, 10: 255 - 
282, 1945. 


Hanna, P. R., Hanna, Jean S., Hodges, R.E., 
and Rudorf, E. H., Jr., Phoneme Grapheme 
Correspondences as Cues to Spelling Improve- 
ment, U.S. Office of Education Cooperative Re- 
search Program Monograph 16, Government 
Printing Office, Washington, D.C., 1966. 


Hays, W. L., Statistics for Psychologists, Holt, 
Rinehart, and Winston, New York, 1965. 


Hodges, R.E., 
Spelling," 
1965. 


“Тһе Psychological Bases of 
Elementary English, 42:629-635, 


Horn, E., Teaching Spelling, American Educa- 
tional Research Association, Department of 


Classroom Teachers Research Pamphlet, М№а- 
tional Education Association, Washington, D.C., 
1963, 


Horn, E., ‘‘Phonetics and Spelling, 
tary School Journal, 57:424-432, 1957. 


Horn, T. D., ‘‘The Effect of the Corrected Test 


on Learning to Spell," Elementary School Jour- 
nal, 47:277-285, 1947, 


Elemen- 


Kottmeyer, W., and Ware, Kay, Spelling Goals, 
Webster Publishing Company, St. Louis, 1960. 


Lindquist, E. F., Design and Analysis of Experi- 
ments in Psychology and Education, Houghton- 
Mifflin, Boston, 1953, 


716. 


18. 


19. 


20. 


21, 


22. 


23. 


. Personke, C., and Yee, А. H., 


91 


McNeil, J.D. , ** Programed Instruction Versus 
Usual Classroom Procedures in Teaching Boys 
to Read," American Educational Research Jour- 
nal, 1:113-119, 1964, 


**A Model for 
Elementary 


Personke, C., and Yee, A.H., 
the Analysis of Spelling Behavior," 
English, 43:278-284, 1966. 


“The Situational 
Choice and the Spelling Program," Elementary 


English, 45:32-37, 40, 1968. 


Platt, J.R., ‘‘Strong Inference,“ Science, 146: 


347-353, 1964. 


“Are Teachers Fair to Boys ?" 
1968. 


Pollack, J. H., 
Today's Health, 46:20-25, 


Rice, J.M., 


The Forum, 23:163-172, 


“The Futility of the Spelling Grind,’ 
1897. 


Solomon, R., 
Design," 
1949. 


**An Extension of Control Group 
Psychological Bulletin, 46:137-150, 


Veldman, D. J., Fortran Programming for the 


Behavioral Sciences, Holt, Rinehart and Winston, 
New York, 1967. 


Yee, A. Н. ,'* The Generalization Controversy in 


Spelling Instruction," Elementary English, 43: 
154-161, 1966. 


THE JOURNAL OF EXPERIMENTAL EDUCATION 
(Volume 37, Number 4, Summer 1969) 


THE USE OF MARKOV CHAINS TO ESTIMATE | 
EFFECTS OF FEEDBACK ON DECISION-MAKING) 


GORDON WELTY 
Chatham College, Pittsburgh 


ABSTRACT 


This research pape: 


r utilizes Billingsley's work on Markov processes in the estimation of behavior of the 
rational decisionmaker, 


There are n levels of 
the stable value $, $ 
++. bn ] 


treatment Ф, including 
T Фф Bor, 

Let [ pij] be a stochastic matrix, 

Pij= 0 and PPY=1, Gj *712,.. n) 


The transition probability of Moving from 
treatment Bi to Ф is Pj. The Probability Pio 


gives the Probability of fin 


ding the process initially 
at treatment leve] $i his can be interpreteq as 
the point at which feedback Occurs, 


The transition Probability Pij isa function of 
an unknown parameter R. Let Pij (R) = R, 
Еф. We propose a test of the hypo! 


thesis that the 
stationary or terminal distribution Я 


of p; i 


fied set Р, P seee n } , under the Conditions that 


n 
zZ Pi Pij = pj (§= 1,2. ,.n), 
1 


By Theorem 5. 4 of Billingsley (2), we derive the 
equation set mi 

P Pi Pij = pj » ріј = 
1 Аја n, 


with Tjj the transition frequences, ^ and и the 


ij, for à; = 


5 а speci- 


counting measure о! 
Ње equations 
the Solution, 


This statistic is a 
esis on P, and if 


У f 
n the state space, For as p 
must be solved iteratively. If ij is 
then the log likelihood statistic 


my 
= ===, 2 i 
ij mi qi 


t 
Ppropriate for testing the hyp? 


2 
(Pij - qij) А 
5 оцаша авы | 


we Suppose ata 


J level! of Confidence a 
tionary distributi 

Permits t 
ability distribut 


, that the wo 
10n of p is the Specified set. es 
mation of R, as for each unique p 
lon p there is one reaction functio! 
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